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FOREWORD 


The  Technologies  for  Skill  Acquisition  and  Retention  Technical  Area  of  the  U.S. 
Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  performs  research 
and  development  in  education  as  part  of  its  program.  A  major  focus  of  this  research  is 
the  development  of  information  on  which  the  Department  of  the  Army  can  base 
decisions  about  its  basic  skills  education  programs.  The  work  described  in  this  report 
was  conducted  under  Project  A794,  Education  and  Training,  cis  part  of  Task  311, 
Improving  Job  Skills  Education  for  Soldiers. 

This  report  describes  work  carried  out  at  the  request  of  the  Chief,  Soldier 
Education  Division,  Total  Artny  Personnel  Agency,  who  requested  that  ARI  evaluate 
the  Job  Skills  Education  Program  (JSEP),  a  computer-based,  job-relevant  curriculum 
designed  to  teach  soldiers  competencies  prerequisite  to  learning  job  skills.  The 
researcher  sampled  JSEP  lessons  for  evaluation  and  reviewed  the  program's  diagnostic 
pretest,  soldier  management  plan,  and  instructor's  manual.  Recommendations  for 
revising  or  improving  these  materials  are  offered. 

This  effort  was  supported  by  the  Soldier  Education  Division,  Total  Army 
Personnel  Agency,  Office  of  the  Deputy  Chief  of  Staff  for  Personnel.  Division  staff 
members  were  provided  with  copies  of  the  report  in  August  1988. 


EVALUATION  OF  THE  JOB  SKILLS  EDUCATION  PROGRAM:  CURRICULUM 
REVIEW 

EXECUTIVE  SUMMARY 


Requirement: 

This  review  of  the  Job  Skills  Education  Program  (JSEP)  curriculum  was  carried 
out  as  part  of  the  overall  evaluation  of  JSEP  to  assess  the  quality  of  instructional  design 
and  to  provide  information  useful  for  further  lesson  revision  and  development. 

Procedure: 

About  22  percent  of  the  total  number  of  JSEP  lessons  were  reviewed.  JSEP  consists 
of  both  lessons  presented  on  the  computer  (on-line  lessons)  and  lessons  in  the  form  of 
paper  materials  (paper-based  lessons).  Both  types  were  analyzed.  In  addition,  JSEP 
supporting  materials,  including  the  JSEP  diagnostic  test,  the  Instructor's  Manual,  the 
Soldier  Management  System:  Features  and  Functions  Guide,  and  a  selected  Learning 
Strategies  lesson  were  reviewed.  It  should  be  noted  that  at  the  time  of  the  review  some 
lessons,  as  well  as  the  manuals,  were  not  yet  in  final  form.  Therefore,  some  of  the 
problems  noted  may  already  be  resolved. 

For  each  selected  prerequisite  competency,  the  review  process  involved  going 
through  the  short  lesson,  the  lesson  test,  and  in  some  cases  the  long  lesson.  (Long 
lessons  were  not  reviewed  as  systematically  as  short  lessons  because  soldiers  went 
through  them  far  less  often.)  Notes  taken  on  the  lessons  were  later  coded  to  reflect  18 
different  problem  categories  that  appeared  during  the  course  of  the  review. 

Lessons  were  reviewed  independently  of  diagnostic  statistics  such  as  student  test 
scores.  However,  the  review  comments  are  intended  in  some  cases  to  help  explain 
results  among  soldiers  using  JSEP. 

Findings: 

The  major  finding  was  that  in  about  half  of  all  JSEP  lessons  reviewed,  deficiencies 
in  teaching  strategy  lead  students  to  learn  skills  that  were  only  superficially  or 
peripherally  related  to  the  performance  of  the  major  targeted  skill  of  the  lesson.  About 
half  of  the  lessons  did  not  teach  underlying  concepts  essential  to  an  understanding  of 
the  skill  but  presented  instead  sets  of  steps  or  rules  that  could  lead  students  to  solve 
problems  without  an  understanding  of  the  processes  involved.  Definitions  of  basic 
terms  and  concepts  essential  to  understanding  lesson  skill  objectives  were  often  not 
given  or  were  put  into  a  form  that  could  mislead  the  student.  Difficulties  with 
language,  organization,  and  practice  and  test  items  (such  as  ambiguities  or  cases  where 
items  did  not  test  appropriate  content)  were  also  noted. 
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JSEP  lessons  were  originally  designed  to  distinguish  between  review  (in  the 
diagnostic  review  or  "short"  lesson)  and  instruction  (in  the  skill  development  or  "long" 
lesson),  assuming  that  a  student  either  once  knew  a  skill  and  had  forgotten  it  or  had 
never  learned  it  in  the  first  place.  [B.  P.  Roberts  and  R.  K,  Branson,  "U.S.  Army  Job 
Skills  Education  Program  (JSEP)  Technology  Transfer  Plan,"  January  15, 1987,  p.  13.] 
However,  it  was  found  that  the  "instruction"  in  some  long  lessons  did  not  differ  from 
the  "review"  in  the  short  diagnostic  review  lesson,  thus  raising  doubts  as  to  the 
instructional  value  of  long  lessons.  Paper-based  lessons  contained  errors  and  significant 
organizational  problems;  in  some  cases  they  were  also  very  lengthy. 

In  sum,  the  major  leanung  benefits  of  the  ctirriculum  are  likely  to  result  from 
systematic  exposure  to  and  review  of  targeted  skills  and  from  the  high  interactiveness 
of  nearly  all  of  the  lessons,  rather  than  from  instructional  design  or  teaching  strategy 
alone. 

The  JSEP  supporting  materials,  prepared  when  the  management  of  JSEP  was  still 
in  the  hands  of  Florida  State  University,  need  careful  rewriting  to  make  them  helpful  to 
users.  This  is  especially  important  in  the  case  of  the  Instructor's  Manual,  because 
budget  constraints  may  mean  that  JSEP  instructors  cannot  count  on  training  or  access  to 
consulting. 

Utilization  of  Findings: 

It  is  recommended  that  these  findings  be  used  to  revise  the  JSEP  curriculiim  as 
follows: 

•  Revise  lessons  to  focus  instruction  on  the  targeted  skill  of  the  lesson,  rather  than 
on  supporting  skills.  In  some  cases,  this  may  mean  putting  more  emphasis  on 
explanation  of  processes  or  procedures  that  underlie  lists  of  steps  or  rules. 

•  Revise  language,  test  items,  and  practice  items  to  eliminate  errors  and  am¬ 
biguities. 

•  Revise  both  short  and  long  lessons  to  include  useful  and  accurate  explanations 
of  imderlying  concepts.  Long  lessons  should  be  revised  so  that  they  do  not 
repeat  matenal  from  short  lessons. 

•  Revise  supporting  materials  so  that  they  are  more  comprehensible  and  helpful 
to  users. 

In  addition,  students  should  be  permitted  more  flexibility  in  their  use  of  JSEP. 
They  should  be  able  to  re-take  lesson'  or  portions  of  lessons,  if  desired,  and  they  should 
be  allowed  to  see  what  their  errors  were  on  tests.  These  changes,  if  made,  would  permit 
JSEP  to  make  better  use  of  the  advantages  inherent  in  computer-based  instruction. 
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EVALUATION  OF  THE  JOB  SKILLS  EDUCATION 
PROGRAM:  CURRICULUM  REVIEW 


Chapter  1.  Introduction 


Purpose 

The  purpose  of  this  review  is  to  provide  commentary  on  the  JSEP  curriculum  from  a 
perspective  emphasizing  instructional  excellence.  This  information  may  be  used  (1)  for 
revision  of  the  lessons  discussed,  (2)  as  a  guideline  for  lesson  analysis  and  revision  in 
other  lessons  that  have  not  been  reviewed,  or  (3)  as  a  source  of  possible  explanation  for 
soldier  outcomes  as  recorded  in  lesson  statistics.  Review  comments  can  also  be  used  to 
account  for  student  reactions  to  particular  lessons  or  parts  of  lessons  as  reflected  in 
survey  responses,  interviews,  personal  notes,  and  classroom  observations.  Ultimately,  the 
value  of  JSEP  as  an  instructional  medium  will  be  measured  by  improvements  in  student 
test  scores  and  job  performance.  However,  at  this  point  it  is  also  useful  to  evaluate  the 
curriculum  solely  from  an  instructional  design  point  of  view,  talcing  into  account  basic 
principles  of  curriculum  design  and  delivery  that  may  affect  student  interaction  with  the 
materials  and  determine,  in  the  long  run,  whether  students  in  fact  learn  sufficiently  well 
from  the  program.  The  focus  in  this  review  will  therefore  be  upon  the  instructional  value 
of  the  lessons  themselves  as  evaluated  independently  of  lesson  statistics  and  student  test 
performance. 


Data  Sources 

The  primary  sources  of  data  for  this  review  are  notes  and  lesson  hard  copy  taken 
during  on-line  analysis  of  JSEP  lessons.  Other  data  sources  include  copies  of  JSEP  paper- 
based  lessons,  soldier  personal  notes  (p-notes),  lesson  diagnostics,  soldier  survey 
comments,  and  field  notes  taken  during  interviews  with  JSEP  students  and  during 
observations  of  JSEP  classrooms.  It  should  be  noted  that  this  review  was  conducted  on 
the  PLATO  version  of  the  JSEP  curriculum,  rather  than  on  the  MICROTICCIT  version. 
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Also,  use  of  a  Macintosh  MacPlus  terminal  may  have  contributed  to  minor  distortion  in 
some  of  the  the  lesson  graphics. 


JSEP:  Brief  Overview 

As  of  this  writing,  JSEP  consists  of  186  lessons  intended  to  teach  the  prerequisite 
competencies  soldiers  need  to  learn  and  perform  their  jobs.  These  lessons  are  of  three 
types:  lessons  that  are  entirely  on-line,  lessons  that  are  entirely  paper-based,  and  lessons 
that  combine  both  formats.  Each  prerequisite  competency  is  covered  by  a  diagnostic 
review  lesson  (or  short  lesson),  and  a  lesson  test.  The  short  lesson  is  intended  primarily 
as  a  review  of  a  skill  that  is  presumed  to  have  been  learned  at  some  point,  but  may  have 
been  forgotten.  For  all  but  34  prerequisite  competencies,  there  is  also  a  skill  development 
lesson  (or  long  lesson)  that  students  take  if  they  fail  the  lesson  test  after  taking  the 
short  lesson.  This  long  lesson  is  intended  to  provide  the  student  with  instruction  in  the 
targeted  prerequisite  competency,  assuming  that  the  student  never  learned  it  in  the  first 
place. 


The  Review  Process 

A  total  of  35  on-line  lessons  and  five  paper-based  lessons  were  reviewed.  This 
reflects  approximately  22  percent  of  the  total  number  of  JSEP  lessons.  The  review 
process  involved  taking  the  lesson  as  a  soldier  would,  going  through  the  short  lesson,  the 
test,  and  in  some  cases  the  long  lesson.  The  time  required  to  get  through  a  lesson  was 
far  longer  than  that  required  by  the  average  soldier  because  of  the  need  to  take  notes,  to 
compare  portions  of  short  lessons  with  long  lessons,  to  compare  test  or  practice  items 
with  lesson  content,  to  compare  different  sections  of  the  same  lesson,  or  to  examine 
feedback  for  wrong  answers  to  practice  questions.  All  the  short  lessons  were  reviewed, 
but  only  certain  long  lessons  or  particular  sections  of  long  lessons  were  reviewed, 
especially  when  the  lessons  had  many  sections,  or  if  they  appeared  to  repeat  material 
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presented  in  the  short  lesson.^  The  review  focused  more  upon  the  diagnostic  lessons  than 
upon  the  long  lessons  simply  because  more  soldiers  are  likely  to  take  the  diagnostic  than 
the  long  lesson,  and  because  the  underlying  rationale  for  JSEP  is  primarily  remediation 
rather  than  instruction.^ 

Lessons  were  selected  for  review  based  on  a  number  of  criteria.  First,  an  attempt 
was  made  to  review  at  least  one  lesson  from  each  different  lesson  series  included  in  the 
common  task  prescription,  since  the  common  task  lessons  are  those  that  all  soldiers  must 
take  while  in  JSEP.  Most  often,  the  particular  lesson  to  be  reviewed  in  a  series  was 
chosen  randomly;  in  some  cases,  more  than  one  lesson  in  a  series  was  reviewed.  Second, 
some  lessons  were  chosen  because  comments  from  the  field  had  indicated  that  such  lessons 
were  "difficult"  or  posed  problems  for  soldiers,  and  review  might  be  able  to  identify  the 
source  of  such  difficulty  for  later  revision.  However,  it  should  be  noted  that  no 
systematic  attempt  to  select  lessons  based  on  diagnostic  reports  was  made,  so  that  the 
sample  of  lessons  reviewed  would  not  biased  in  favor  of  either  "poor"  or  "good"  lessons. 

The  lessons  reviewed  represent  only  a  sample  of  the  total  number  of  JSEP  lessons. 
Primarily  because  of  the  time  consuming  nature  of  the  on-line  review  process,  it  would 
have  been  impossible  to  review  every  lesson.  However,  a  sufficient  number  of  lessons 
have  been  reviewed  to  enable  general  patterns  across  lessons  to  emerge  in  analysis. 

Over  the  sbe-month  period  during  which  the  on-line  lesson  review  was  conducted, 
Florida  State  University  continued  to  review  and  revise  JSEP  lessons.  Thus,  at  the  time  a 
lesson  was  reviewed  it  may  not  have  been  in  its  present  implemented  form.  This  may  be 
significant  as  far  as  some  of  the  specifics  of  lesson  content  and  presentation  are 
concerned  (e.g.,  errors  in  practice  or  test  items;  language  and  typographical  errors). 
However,  the  revisions  performed  by  Florida  State  University  generally  did  not  include 
large  scale  rewriting  of  lessons  involving  changes  in  design,  organization,  content,  or 


^The  following  lessons  were  reviewed:  lb,  Ic*,  Id*,  Ig*,  2a,  3a*,  3c*,  4a*,  5e,  6b,  7a, 
8d,  9a,  9d,  11a  (p),  12e*,  12h*,  13c,  14d,  14f*,  15d*,  16c*,  16f,  16h*,  25d  (p),  26a,  26e, 

27c*,  27e*,  28b  (p),  29b*,  29c,  30a,  31a,  32e*,  33a  (p),  34d,  35d  (p),  36c,  41b.  The  asterisk 
indicates  that  the  long  lesson  was  also  reviewed.  (P)  indicates  the  lesson  is  paper-based. 

^B.  P.  Roberts  and  R.  K.  Branson,  "U.S.  Army  Job  Skills  Education  Program  (JSEP) 
Technology  Transfer  Plan,"  January  15, 1987,  P.  13. 
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approach.  Thus  the  majority  of  concerns  in  this  review  which  relate  to  the  broader 
aspects  of  lesson  design  may  be  taken  as  still  valid. 


Lesson  Evaluation  Criteria 

The  evaluation  criteria  upon  which  this  review  is  based  were  derived  from  a 
combination  of  principles  of  instructional  design  and  basic  skills  applied  to  field-based 
lesson  analysis.  Such  a  process  identified  the  following  categories  and  characteristics  of 
lessons  that  could  be  considered  important  areas  for  evaluation: 

1.  Aim  or  purpose. 

•  Does  the  lesson  have  a  stated  purpose  which  is  actually  carried  out  in  the 
lesson?  Does  the  lesson  content  match  the  stated  purpose? 

2.  Organization. 


•  Do  the  sections  in  the  lesson  have  a  recognizable,  sensible  order?  Are 
subsections  clearly  and  logically  ordered? 

3.  Concept  presentation. 

•  Does  the  lesson  state  or  teach  the  underlying  concept  behind  the  skills  the 
soldier  is  asked  to  learn?  Is  this  concept  adequately  stated  and  explained? 

•  Does  the  lesson  explain  why  given  skills  or  tasks  must  be  performed,  or 
explain  the  rationale  for  certain  procedures  so  that  the  soldier  can  understand 
why  he/she  is  asked  to  do  something? 

4.  Tasks/Skills  presentation. 

•  What  types  or  levels  of  skills  are  taught?  Are  these  the  most  appropriate  for 
performing  given  tasks  or  solving  given  problems? 

•  How  are  skills  taught?  If  "steps"  are  used,  is  this  the  best  method?  Are  all 
necessary  steps  included  and  all  unnecessary  steps  avoided? 
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•  Are  examples  well-chosen?  Do  they  adequately  illustrate  how  steps  or 
methods  are  used  in  solving  realistic  problems? 

•  What  use  is  made  of  prior  or  assumed  background  knowledge?  Are  such 
assumptions  appropriate? 

5.  Short  Lesson-Long  Lesson  Relationship. 

•  How  are  these  lessons  related?  Are  they  similar  to  or  different  from  each 
other  in  content,  approach,  practice? 

•  How  well  are  the  stated  review  and  instructional  purposes  of  each  lesson  met? 
Which  lesson  is  more  effective? 

6.  Language. 

•  Are  all  statements,  explanations,  problems,  etc.,  worded  in  the  clearest, 
simplest  way?  Are  there  ambiguities? 

•  Is  the  terminology  used  in  lessons  consistent  both  within  and  across  lessons? 

7.  Practice  Items. 

•  Are  these  placed  at  the  appropriate  moments  in  a  lesson?  Do  they  illustrate 
the  intended  skills  or  concepts  taught  in  the  particular  lesson/lesson  section? 

•  Are  they  well  designed  to  enable  students  to  practice  relevant  skills? 

•  Are  items  balanced  to  reflect  emphasis  given  in  a  lesson  on  different  skills  or 
content  areas? 

•  Do  they  provide  appropriate  and  helpful  feedback  -  i.e.,  feedback  that  is 
error-free  as  well  as  explanatory  or  illustrative? 

•  Is  sufficient  indication  given  of  the  format  required  for  answers?  Are 
answers  that  require  much  accurate  typing  avoided? 

•  How  are  practice  items  related  to  test  items?  Is  there  sufficient  match? 

•  Do  practice  items  give  students  the  chance  to  advance  to  use  of  skills  in 
higher  level  contexts?  Is  there  adequate  provision  for  advancement  to  higher 
levels  or  for  application  of  skills  in  different,  related  situations? 
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8.  Test. 


•  Are  questions  clearly  worded  so  that  the  student  knows  what  is  being  asked? 

•  Is  enough  indication  given  to  the  student  as  to  the  required  answer  format 
(in  both  English  usage  as  well  as  in  mathematical  specifications)? 

•  Do  the  test  items  reflect  the  balance  of  material  covered  in  the  lesson? 

•  Are  accompanying  graphics  easy  enough  to  read? 

•  Do  test  items  in  fact  test  the  skills  students  have  been  taught  in  the  lesson? 
What  do  they  test? 

9.  General  Issues. 

•  Presentation;  What  is  the  quality  of  the  screen  display,  graphics,  diagrams, 
paper  materials? 

•  User  choice:  What  options  do  students  have  in  going  through  lessons?  How 
"flexible"  is  the  program  in  responding  to  student  needs  or  choices? 

•  Subjective  appeal:  What  are  some  of  the  appealing  features  of  the  curriculum 
design? 


In  addition  to  evaluating  the  on-line  curriculum  in  prerequisite  competencies,  this 
review  also  considers  JSEP  supporting  materials.  These  include  a  JSEP  Learning  Strategies 
lesson,  the  JSEP  Instructor’s  Manual,  the  Soldier  Management  System:  Features  and 
Functions  Guide,  and  the  JSEP  Test.  The  review  first  describes  and  illustrates  specific 
problems  found  in  JSEP  lessons,  and  then  moves  to  consideration  of  general  curricular 
issues,  supporting  materials,  and  recommendations. 
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Chapter  2.  Specific  Lesson  Problems 


Problems  in  JSEP  lessons  fall  into  three  general  categories:  conceptual  level 
problems,  problems  with  presentation  or  delivery,  and  problems  in  short/long  lesson 
compatibility  (including  tests).  For  each  problem  area,  examples  illustrating  the  problem 
are  drawn  from  the  lessons  reviewed.  Although  in  many  cases  the  examples  given  here 
illustrate  more  than  one  problem  area,  for  the  sake  of  clarity,  the  discussion  focuses  on 
only  one  type  of  problem.  Table  2.1  illustrates  the  lessons  reviewed  and  the  categories  of 
problems  found  in  each.  The  percent  of  lessons  manifesting  each  type  of  problem  is  also 
indicated. 


Conceptual  Problems 

Problem  area  1.  The  lesson  does  not  teach  what  it  intends  to  teach:  hut  rather 
teaches  skills  focused  on  the  surface  features  of  a  given  problem.  Although  the  intention 
of  the  lesson  is  to  teach  a  particular  concept-based  skill,  the  skills  actually  taught  are 
peripheral  to  the  underlying  skill.  In  some  cases  the  skiil(s)  actually  taught  "mimic" 
performance  of  the  intended  skill  -  i.e.,  they  bring  the  student  to  the  same  result,  yet 
the  underlying  conceptual  basis  for  performance  and  understanding  is  lacking.  Thus 
students  can  be  taught  how  to  solve  problems  without  necessarily  achieving  any 
understanding  of  the  basic  concepts  involved  in  the  process.  If  it  is  true  that  lower 
ability  students  often  lack  a  conceptual  model  to  facilitate  learning,  one  of  the  major 
objectives  of  JSEP  lessons  ought  to  be  to  provide  the  necessary  conceptual  model  or 
information  to  allow  the  student  to  learn  a  skill,  not  merely  just  to  perform  it  in  a 
mechanical  fashion  so  that  the  result  is  correct.  Examples  of  this  problem  follow. 

Lesson  34d,  "Labeling  the  Parts  of  an  Outline.”  The  purpose  of  this  lesson  is  clearly 
stated  at  the  outset:  ". . .  to  identify  each  part  of  an  outline  as  major  topic,  subordinate 
topic,  or  detail  and  to  show  how  these  are  organized  to  form  the  whole  topic."  [Note: 


the  phrase  . .  to  form  the  whole  topic"  is  somewhat  vague.]  Thus,  the  major  "skill" 
intended  by  the  lesson  is  one  of  understanding  the  relationship  between  major  topics, 
subordinate  topics,  and  details.  In  order  to  label  an  outline  correctly  the  student  must 
understand  what  levels  of  abstraction,  generalin'.  and  specificity  are,  and  how  these  are 
reflected  in  the  outline.  The  lesson  in  fact  does  not  teach  these  skills.  Rather,  it 
teaches  the  recognition  and  manipulation  of  the  outline  code  without  teaching  soldiers  how 
to  determine,  for  example,  if  a  topic  is  major  or  subordinate,  or  how  to  see  the 
relationship  between  the  two.  This  is  clearly  reflected  in  the  steps  which  soldiers  are 
told  to  perform  in  order  to  label  an  outline: 

Step  1.  Alternate  and  indent  numbers  and  letters  to  make  a  code. 

Step  2.  Identify  major  topics,  subtopics,  and  details. 

Step  3.  Use  the  code  to  label  the  major  topics,  subtopics,  and  details. 

The  fact  that  this  lesson  docs  not  teach  what  it  intends  to  teach  is  evident  in  the 
highly  unnatural  and  indeed  impossible  task  of  performing  step  one  independently  of,  and 
prior  to,  step  two.  How  can  one  even  begin  to  know  what  the  "code"  looks  like  without 
first  knowing  how  the  elements  to  be  contained  in  the  outline  are  related?  In  reality  the 
outlining  process  entails  simultaneous  coding  and  identifying  relations  between  topics. 

Here,  the  process  of  making  a  code  is  separated  from  the  process  of  identification  of 
types  of  topics.  This  is  a  case  not  only  of  an  inappropriate  focus  on  the  surface  features 
of  a  skill,  but  also  of  problem  area  2,  overspecification  or  reduction  of  skills  (to  be 
discussed  in  the  following  section.) 

The  lesson  adequately  teaches  students  the  elements  of  the  code  itself;  indeed,  it 
provides  extensive  practice  in  recognizing  the  code  pattern.  The  practice  items  take  the 
soldier  from  filling  in  some  missing  elements  of  a  code  to  generating  a  code  for  a  pre¬ 
organized  list,  an  effective  way  of  familiarizing  the  student  with  the  code  elements.  But 
there  is  no  instruction  in  how  to  determine  relationships  among  topics.  This  is  the  real 
conceptual  problem  beneath  the  coding  process,  and  it  is  never  addressed  in  the  lesson. 

Lesson  Ic,  "Ordering  Numbers-Intervals."  The  objective  of  this  lesson  is  to  teach 
soldiers  how  to  sequence  numbers,  including  decimal  numbers  and  negative  numbers. 
However,  the  lesson  does  not  in  fact  teach  students  to  sequence  numbers  --  that  is,  to 
recognize  relative  value  (the  underlying  "intended"  concept).  Rather,  it  teaches  students 
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peripheral  identification  or  recognition  skills  that  ultimately  have  little  to  do  with  this 
essential  skill.  For  example,  in  the  long  lesson  section  "Sequencing  Positive  Numbers  with 
Decimals"  the  focus  is  placed  upon  the  skill  of  labelling  place  values,  primarily  by  noticing 
where  the  decimal  point  is  positioned: 

(p.  13)  Positive  decimal  numbers  may  be  a  little  more  complicated  because  you 
must  pay  close  attention  to  where  the  decimal  point  is  placed.  For  example, 
look  at  this  series: 

3.021  3.120  3.012  3.2 
The  largest  number  in  the  set  is  3.2. 

In  order  to  determine  the  correct  answer,  you  will  need  to  know  the  numbers 
and  the  place  value  of  the  numbers. 

As  a  matter  of  fact,  the  decimal  point  is  "placed"  in  exactly  the  same  position  in  all 
four  cases  --  after  the  whole  digit.  Thus,  paying  close  attention  to  where  the  decimal 
point  is  placed  has  nothing  to  do  with  determining  the  relative  value  of  these  numbers. 

The  practice  items  in  this  section  focus  merely  on  labeling  place  values  --  pages  14, 
15, 16,  17  all  give  practice  in  labelling  and  identifying  the  tenths  place,  the  hundredths 
place,  etc.  The  rule  given  on  p.  17,  for  example,  states: 

Two  place  decimals  are  written  with  the  decimal  point  (.)  followed  by  two 
digits.  Ninety  nine  hundredths  is  99  parts  of  100  parts. 

True  enough,  but  how  is  this  information  going  to  be  helpful  to  the  student  who 
wants  to  compare  numbers?  Although  this  is  a  component  skill  in  the  process  of 
recognizing  relative  value,  it  is  not  the  same  thing  as  being  able  to  compare  place  values. 

In  fact,  the  lesson  provides  little,  if  any,  explicit  relative  place  value  comparison  practice. 
(For  example,  students  could  be  taught  to  compare  the  tenths  place  first  --  with  an 
explanation  of  how  tenths  are  "larger"  portions  than  hundredths,  etc.) 

Lesson  15d,  "How  to  Use  a  Protractor."  Another  example  of  this  process  of  "missing  the 
main  point"  behind  a  skill  is  found  in  this  long  lesson.  On  page  11  the  lesson  asks  the 
soldier  for  the  largest  number  on  the  degrees  scale  of  the  protractor.  The  largest 
possible  number  of  degrees  is  360;  however,  the  largest  printed  number  is  355,  which  is 
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the  answer  accepted  by  the  program.  The  point  of  the  question  should  be  -  or  in  fact  is 
--  to  teach  the  student  that  the  largest  measure  on  the  protractor  corresponds  to  that  of 
a  circle,  360  degrees,  not  that  355  degrees  is  the  largest  printed  value.  Then  on  page  12, 
after  the  soldier  has  been  told  that  355  is  the  largest  number  on  the  degrees  scale, 
he/she  is  told:  "Notice  that  the  degree  scale  measures  from  0  to  360.  Note:  the  360  and 
0  marks  are  the  same."  A  misfocus  on  the  "surface"  features  of  the  protractor  has 
replaced  the  underlying  concept  which  is  intended  to  be  taught  Fortimately,  in  this  case 
the  situation  is  remedied  at  a  later  point  in  the  lesson. 

Problem  area  2.  The  lesson  teaches  a  skill  as  a  series  of  steps  or  rules  that 
substitute  for  necessary  instruction  in  the  underlying  concept  behind  the  skill.  Often, 
these  steps  or  rules  overspecify  the  task,  breaking  it  down  into  component  skills  that 
could  be  taught  and  performed  more  naturally  as  a  unit. 

The  step  or  rule-based  approach  to  instruction  is  found  in  nearly  all  JSEP  lessons.  In 
some  cases,  presenting  a  list  of  steps  helps  to  clarify  the  tasks  that  need  to  be  done  to 
perform  a  skill;  in  other  cases,  however,  the  rules  contribute  to  confusion  and  make  the 
task  more  difficult  than  it  really  is.  When  rules  or  steps  multiply,  the  student  has  more 
chances  to  make  an  error  in  carrying  out  a  procedure.  This  is  especially  true,  as  in  the 
following  example,  when  the  rules  or  steps  are  linked  to  the  surface  features  of  a  task 
rather  than  underlying  ideas. 

Lesson  Ig,  "Rounding  Numbers."  In  this  lesson  the  skill  of  rounding  is  taught  as  sets  of 
rules  that  specify  what  must  be  done  for  each  separate  case  of  rounding  to  a  specific 
number,  rather  than  through  the  application  of  a  generalized,  concept-based  rule  that  can 
be  used  for  any  case  where  rounding  is  required.  Students  are  given  multiple  sets  of 
rules  to  follow  for  different  cases  of  rounding  to  the  nearest  5,  nearest  25,  nearest  50, 
etc.: 


(p.  12)  Rounding  Numbers  to  the  nearest  5. 

Sometimes  you  must  round  to  the  nearest  5.  For  example,  one  of  your  tasks 
might  be  to  round  off  the  fuze  setting  to  the  nearest  5. 

If  the  fuze  setting  is  26,  you  would  round  it  to  25. 

If  the  fuze  setting  is  23,  you  would  also  round  it  to  25. 

But  if  the  fuze  setting  is  21,  you  would  round  it  to  20. 
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On  page  13,  the  lesson  lists  five  rules  for  rounding  to  the  nearest  5.  These  rules 
require  the  student  to  "learn"  what  to  do  with  specific  cases,  instead  of  learning  a 
general  concept  that  would  allow  him/her  to  deal  with  any  case. 

Rule  1.  Numbers  ending  in  0  or  5  are  OK  the  way  they  are. 

Rule  2.  Numbers  ending  in  1  or  2  round  down  to  the  nearest  number  that  ends 
in  0. 

Rule  3.  Numbers  ending  in  3  or  4  round  up  to  the  nearest  number  that  ends  in 
5. 

Rule  4.  Numbers  ending  in  6  or  7  round  down  to  the  nearest  number  that  ends 
in  5. 

Rule  5.  Numbers  ending  in  8  or  9  round  up  to  the  nearest  number  that  ends  in 
0- 

Similar  sets  of  rules  -  all  of  which  focus  student  attention  on  the  surface  aspects 
(in  this  case,  digits)  of  the  given  problem  rather  than  on  the  underlying  concept  behind 
rounding  --  are  given  for  each  case  of  rounding  to  5,  to  25,  to  50,  to  100.  That  trans¬ 
lates  into  an  enormous  number  of  rules  for  the  soldier  to  remember  and  work  with.  This 
is  a  highly  impractical  way  to  teach  the  skill  of  rounding;  indeed,  lists  of  rules  appear  to 
substitute  for  an  adequate  explanation  of  the  concept  of  rounding.  Such  an  explanation 
ought  to  at  least  contain  the  idea  that  the  basic  notion  behind  rounding  involves  how  to 
decide  to  which  multiple  a  given  number  is  closer.  The  only  time  this  point  is  made 
comes  in  the  feedback  to  the  practice  question  on  page  22,  when  the  lesson  asks,  "Is  67 
closer  to  50  or  75?"  Had  students  been  given  a  general  concept-based  method  for 
determining  which  of  two  multiples  is  closer  (such  as  comparing  differences,  looking  for 
the  halfway  point,  etc.),  this  multiplication  of  rules  could  have  been  avoided.  Moreover, 
in  the  long  lesson  (where  one  would  expect  more  "instruction")  the  same  rules  are 
repeated  over  again,  still  without  an  adequate  conceptual  framework.  [This  lesson  is  an 
example  of  problem  area  15,  identical  approach  in  long/short  lessons.] 

Lesson  15d,  "How  to  Use  a  Protractor."  In  this  lesson,  the  steps  to  be  followed  in  using 
a  protractor  are  overspecified.  In  the  long  lesson,  section  B,  Step  3  lists  as  two  separate 
steps  what  would  more  naturally  and  easily  be  performed  as  one:  (p.  26) 

3.  Two  steps  to  line  up  protractor  with  the  angle: 

1.  line  up  reference  line  with  line  1. 

2.  line  up  the  vertex. 
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Ten  pages  later  (p.  36)  the  soldier  is  finally  told,  "When  you  line  up  the  reference  line 
with  line  1,  you  can  make  sure  the  index  mark  is  lined  up  at  the  same  time."  Over¬ 
specification  such  as  this,  unless  remedied  by  pointing  out  how  separate  steps  can  be 
collapsed,  contributes  to  holding  students  back  at  a  "basic"  competency  level  rather  than 
encouraging  them  to  advance  to  a  higher  level  of  skill  performance. 

Lesson  16h,  "Solve  Word  Problems  Involving  Any  Mathematical  Process."  This  lesson  relies 
on  a  set  of  rules  that  reduce  solving  a  problem  to  a  mechanical  process  of  pattern 
matching.  These  rules  substitute  for  a  concept-based  approach  to  teaching  the  target 
skill.  In  lesson  16h  the  primary  "step"  in  the  process  is  identified  as  "Select  the  right 
formula."  However,  this  process  is  reduced  to  a  mechanistic  matching  up  of  letters  of  the 
quantity  words  in  the  problem  with  letters  in  formulas.  This  does  not  allow  a  student  to 
move  toward  a  more  "advanced"  level  of  formula  selection  based  on  the  perceived  relation¬ 
ships  between  variables  and  knowledge  of  which  formulas  use  such  relationships.  For 
example,  the  long  lesson  Section  C  (devoted  to  formula  selection)  asks  the  student: 

What  formula  should  be  used  to  solve  this  one? 

Calculate  the  (P)erimeter  of  a  square  given  that  its  (S)ides  each  measure  10 

inches.  Touch  the  box  with  the  correct  answer. 

P=  4S  P=  tD 
A=S2  A=  R2 

An  incorrect  answer  elicits  the  warning,  "Watch  the  initial  letters  of  the  quantities 
in  the  problem." 

The  section  concludes: 

It’s  easy  to  select  the  appropriate  formula  to  solve  a  word  problem.  Just 

choose  the  one  which  matches  the  initial  letters  of  the  quantities  or  units  of 

the  problem. 

(R)ate  of  Speed  =  50  mph.  (D)istance  =  ?  (T)ime  =  2  hrs. 

D  =  RT 

In  all  the  practice  questions,  formulas  appear  ready-made  and  available  for  the 
student  to  match  letters  up  with."  What  about  cases  in  which  no  formulas  are  readily 
available?  Is  this  not  a  more  realistic  situation?  This  sort  of  instruction  does  not  really 
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teach  the  student  to  solve  word  problems;  it  merely  teaches  him  or  her  to  notice  letters 
in  formulas.  The  steps  given  do  not  prepare  the  student  for  realistic  situations  in  which 
knowledge  of  basic  mathematical  relationships  is  involved  in  solving  problems. 

Problem  area  3.  Concept  explanations  for  procedures  or  processes  are  incomplete  or 
lacking.  In  most  cases  this  translates  into  telling  students  to  perform  a  step  in  a  process 
without  explaining  why  the  step  is  necessary  or  useful,  or  without  explaining  how 
particular  problem-solving  procedures  or  elements  work.  Examples  follow. 

Lesson  16f,  "Solve  Conversion  Problems."  This  lesson  begins  with  a  statement  of  the  rule 
i  to  be  followed. 

(p.2)  To  convert  from  one  unit  of  measurement  to  another,  multiply  by  the 
appropriate  conversion  factor.  The  conversion  factor  converts  one  unit  to 
another,  for  example,  square  feet  to  square  inches. 

[Example]  What  is  the  square  inches  equivalent  of  20  square  feet?  The 
conversion  factor  is  144. 

[144  what?  What  does  this  144  tell  the  student  about  the  relationship  between  the 
units  in  this  problem?  Where  does  this  144  come  from?  What  are  "conversion  factors"  in 
the  first  place?  No  explanation  is  given.] 

20  sq  ft  X  144  =  2880  sq  in 

old  unit  conversion  new  unit 
factor 

There  are  two  problems  with  this  rule:  (1)  there  is  no  real  explanation  of  where 
conversion  factors  come  from  or  how  they  are  derived,  and  (2)  it  is  not  true  that  one 
always  multiplies  to  get  the  answer,  for  later  examples  require  the  student  to  divide. 
given  the  form  in  which  the  problem  is  stated. 

Telling  students  that  conversion  factors  "convert  from  one  unit  to  another" 
contributes  little  to  a  student’s  understanding  of  the  conversion  process.  There  is  no 
explanation  of  how  these  conversion  factors  work.  Furthermore,  students  are  usually 
^  given  the  appropriate  conversion  factor  for  each  problem,  so  there  is  no  need  to  think 

through  what  is  actually  happening  during  the  conversion  process.  These  ready-made 
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conversion  factors  do  not  in  fact  teach  the  student  anything;  they  merely  enable  him  or 
her  to  solve  problems  mechanically  without  understanding  the  process.  There  is  only  one 
indication  of  the  source  of  conversion  factors  in  the  lesson.  On  page  3,  when  students 
must  determine  the  yard  equivalent  of  100  inches,  the  lesson  says,  "One  inch  =  .028  yards 
so  the  conversion  factor  is  .028."  This  is  the  only  "explanation"  for  the  source  of 
conversion  factors;  even  this  does  not  explicitly  get  at  the  numerical  and  conceptual 
relationship  between  yards  and  inches. 

A  related  problem  concerns  discrepancies  in  the  ways  conversion  factors  are 
presented.  In  some  cases  units  are  given,  in  others  they  are  not.  For  example: 

What  is  the  meter  equivalent  of  50  yards? 

(conversion  factor  =  .914) 

What  is  the  gallon  equivalent  of  132  quarts? 

(1  gallon  =  4  quarts) 

How  many  kilometers  equal  100  mi? 

(use  conversion  factor  1.6) 

In  the  first  example,  there  is  no  indication  of  the  relationship  between  meters  and  yards. 
In  the  second,  the  conversion  factor  is  presented  in  an  entirely  different  form  requiring 
the  student  either  to  "convert"  the  conversion  factor  to  the  appropriate  form,  or  to  divide 
to  get  the  answer,  contrary  to  the  multiplication  rule  given  at  the  beginning  of  the 
lesson.  In  the  third  example,  units  are  again  missing.  Unless  students  are  provided  with 
some  explanation  of  how  conversion  factors  are  derived  and  how  they  work,  problem 
solving  is  reduced  to  a  mechanical  process  that  does  not  require  students  to  really 
understand  what  is  going  on.  A  student’s  comment  (p-note)  about  this  lesson,  "I  can 
solve  the  problems  without  understanding  the  process"  is  thus  exactly  on  target. 

Lesson  15d,  "How  to  Use  a  Protractor."  This  lesson  also  lacks  an  explanation  for  a 
procedure.  On  page  18  the  examples  show  the  protractor  alined  vertically,  but  students 
are  not  told  if  this  is  necessary,  nor,  if  it  is,  why.  This  explanation  would  make  good 
sense,  given  that  this  particular  alinement  occurs  repeatedly.  On  page  20,  the  lesson 
states  "You  can  use  a  protractor  to  plan  an  azimuth:  just  aline  the  protractor  on  a 
parallel  to  a  North-South  line  with  the  index  mark  where  you  are  located."  But  there  is 
no  explanation  of  whether  this  is  necessary,  or  why. 


Lesson  14f,  "Multiplying  and  Dividing  Fractions."  In  this  lesson  some  explanations  are 
incomplete  or  misleading.  On  page  7,  students  are  told  how  to  multiply  a  whole  number 
by  a  fraction: 


"To  multiply,  convert  the  whole  number  to  an  improper  fraction,  (an  improper 

fraction  has  a  larger  number  in  the  numerator  than  the  denominator.)  " 

Indeed,  but  the  point  is  that  for  whole  numbers  the  denominator  will  always  be  1,  not 
that  the  numerator  is  larger  than  the  denominator.  This  is  not  specified. 

Moreover,  both  the  diagnostic  and  the  long  lesson  omit  an  explanation  of  how  to 
convert  a  mbced  number  to  an  improper  fraction,  a  skill  essential  to  the  objective  of  the 
lesson.  The  explanations  of  basic  terms  which  come  at  the  beginning  of  the  long  lesson 
are  complete  for  some  terms  (for  example,  for  "reducing  a  fraction"  a  detailed  step-by- 
step  explanation  of  the  process  is  given)  but  incomplete  for  others.  (For  example,  the 
explanation  for  "improper  fraction"  merely  states  that  it  can  be  reduced  to  a  mixed 
number,  and  illustrates:  3/2  =  1  1/2  or  8/2  =  4.  And  for  "mixed  number,"  the 
lesson  simply  states,  "A  whole  number  and  a  fraction  written  side  by  side.") 

Problem  area  4.  Explanations  of  major  underlying  concepts  behind  targeted  skills  of 
the  k.s.son  arc  lacking,  incomplete,  or  misleading.  Thus,  the  lesson  lacks  an  adequate 
focus  on  the  conceptual  level  of  explanation  underlying  skill  performance.  This  was  the 
most  common  problem  identified  in  the  lessons. 

Lesson  25d,  "Finding  Main  Ideas"  (paper-based).  The  objective  of  this  lesson  is  to  teach 
soldiers  to  locate  main  ideas.  However,  the  definition  of  main  idea  around  which  the 
whole  lesson  revolves  is  seriously  deficient,  and  leads  to  confusion  later  on  in  the  lesson. 
The  definition  for  main  idea  given  in  the  lesson  is: 

"The  main  idea  is  a  summary  of  important  facts  or  actions." 

However,  main  ideas  are  not  "summaries;"  they  are  more  like  one-  or  two-sentence 
statements  of  the  guiding  concepts  or  notions  present  in  a  chunk  of  text.  Even  if  they 
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were  "summaries,"  the  inadequacy  of  this  definition  becomes  evident  when  students  are 
told  that  they  can  locate  "main  ideas"  in  the  table  of  contents  of  a  manual: 

"Manuals  are  divided  into  chapters  and/or  sections.  The  main  idea  of  each  is 

listed  in  the  table  of  contents." 

If  main  ideas  are  summaries,  the  inadequacy  of  this  definition  becomes  obvious,  for 
"summaries"  are  not  found  in  the  table  of  contents.  Main  ideas,  moreover,  are  not  "listed 
in  the  table  of  contents;"  "topics"  or  "subjects"  would  be  a  more  appropriate  label  to  use. 

In  this  lesson  an  obvious  attempt  to  simplify  the  concept  of  main  idea  results  in 
conceptual  confusion.  The  notion  of  "main  idea"  actually  becomes  more  difficult  for 
students  to  use  and  understand,  because  now  what  are  in  fact  conceptually  distinct 
notions  of  summary,  topic,  main  idea,  (and,  later  on,  heading)  have  been  confounded.  This 
is  further  emphasized  by  the  way  in  which  the  main  idea  is  initially  presented  as  some¬ 
thing  one  gets  through  reading  a  text,  and  then  as  something  identifiable  through  other 
features  such  as  a  table  of  contents  or  an  index.  The  lesson  tells  students  that  even 
"illustrations  and  tables  . . .  will  help  you  discover  the  main  idea."  (Main  idea  of  what?) 
This  list  of  features  includes  "topic  sentences  of  paragraphs"  in  the  same  breath  as  tables 
of  contents,  further  confounding  the  text-based  notion  of  main  idea  with  the  artificial 
notion  of  main  idea  as  summary  or  topic.  The  main  idea  concept  and  the  notion  of 
locating  information  topically  or  categorically  have  been  confused. 

This  confusion  gets  worse  as  the  student  moves  through  the  lesson.  On  page  12  the 
lesson  describes  "page  format"  as  a  source  of  main  ideas;  the  lesson  states  that  to 
determine  what  the  main  idea  is  one  should  look  for  format  cues  such  as  centering  and 
dark  type.  But  the  feedback  for  the  answer  to  the  practice  question  is  not  based  on 
format  cues,  but  upon  a  conceptual  understanding  of  main  idea:  "This  is  the  first  page  of 
a  description  of  a  procedure,  so  the  purpose  of  this  procedure  ...  is  the  main  heading." 
[Nothing  is  said  at  all  here  about  format  in  making  this  decision.] 

Moreover,  in  this  example,  main  idea  is  confused  with  main  heading.  This  confusion 
continues  on  page  14,  where  a  question  asks:  "What  are  the  headings  on  this  page?" 
[Suddenly  we  have  lost  the  main  heading  idea  and  are  looking  for  regular  headings.]  The 
explanation  then  tells  the  student  that  there  are  three  main  headings.  [But  if  these  are 


main  headings,  where  are  the  regular  headings?]  Then,  the  lesson  brings  in  the  notion  of 
section,  stating  "Each  section  has  its  own  main  idea." 

A  question  is  then  posed:  "What  are  the  paragraph  headings?"  But  the  answer  given 
on  the  following  page  gives  section  headings,  not  paragraph  headings.  Sections  have  now 
been  confused  with  paragraphs;  this  confusion  gets  worse  on  page  17,  where  students  are 
told  that  main  ideas  are  located  in  the  topic  sentences  of  paragraphs.  [They  have  just 
been  told  that  main  ideas  are  located  in  headings.) 

The  misleading  explanations  and  concept  confusion  present  so  far  in  this  lesson  lead 
inevitably  to  the  explanation  on  page  19: 

The  main  idea  for  subparagraph  "a"  by  itself  is  found  in  the  first  sentence  of 

the  subparagraph.  The  sentence  [sic]  in  (1)  and  (2)  support  the  main  idea  in 

subparagraph  "a."  The  main  idea  for  "b"  is  found  in  its  only  sentence. 

Later,  main  ideas  are  identified  with  captions. 

Overall,  the  lesson  lacks  an  adequate  and  clear  presentation  of  the  concept  of  main 
idea,  leading  to  inconsistent  and  confusing  explanations. 

Lesson  Ig,  "Rounding  Numbers."  The  lesson  gives  the  following  definition  of  rounding 
(page  2): 

"What  is  rounding?  Rounding  means  nai  being  exact  when  using  numbers." 

First  of  all,  this  is  not  a  definition  of  rounding;  it  does  not  describe  the  process;  rather 
it  describes  an  outcome  of  the  process.  Furthermore,  this  statement  is  misleading  because 
it  implies  that  rounding  is  an  imprecise  process  when  in  fact  the  rules  governing  it  that 
are  presented  in  the  lesson  are  very  precise. 

The  lesson  then  moves  to  a  definition  and  illustration  of  whole  numbers  and  place 
value,  but  without  any  mention  at  all  of  what  it  means  to  "round  to  the  nearest  x."  The 
heart  of  the  rounding  process  involves  recognition  and  understanding  of  what  ". . .  to  the 
nearest  x"  involves,  but  this  concept  is  not  discussed.  However,  it  could  be  illustrated  in 
a  way  that  compares  the  results  when  one  number  is  rounded  to  the  nearest  5, 10,  25, 
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etc.  The  lack  of  an  adequate  treatment  of  basic  concepts  involved  in  rounding  is  what 
leads  to  the  necessity  of  presenting  the  multiple  sets  of  specific  rules  for  rounding  to  the 
nearest  5,  25,  etc.,  that  are  found  in  the  rest  of  the  lesson. 


Lesson  11a,  "Identify  Shape  and  Position  Terms”  (paper).  In  this  lesson  misleading 
definitions  of  basic  concepts  lead  to  confusion  and  logical  contradictions.  The  problem 
starts  with  the  definition  and  description  of  circles  on  page  4: 


A  circle  is  a  single  curved  line  in  which  every  point  is  an  equal  distance  from 
a  center  point. 


[According  to  this  definition. 


could  also  be  called  a  circle.] 


Circles  drawn  on  paper  are  two-dimensional. 

[Are  there  in  fact  any  "three-dimensional"  circles?  No,  but  the  text  is  not  getting  at  the 
main  point,  which  is  distinguishing  two-dimensional  from  three-dimensional  representa^ 
tions.j 


[1.]  Circles  are  round,  but  not  all  round  things  are  circles.  [2.]  Apples  and 
oranges  are  not  circles.  Also,  circular  objects  are  often  solid,  or  three- 
dimensional,  such  as  a  ball. 

(page  5) 

[3.]  Things  that  are  round  are  circular,  or  are  shaped  like  spheres,  circles, 
globes,  or  balls.  [4.]  An  object  does  not  need  to  be  a  perfect  sphere  to  be 
called  round. 


[Numbers  in  brackets  have  been  added  for  ease  of  reference.] 


In  addition  to  being  redundant  and  confusing,  these  passages  are  also  contradictory. 
According  to  the  above  logic,  an  orange  could  be  called  round.  [4.]  Then,  given  [3],  an 
orange  could  be  called  circular.  This  leads  to  a  contradiction  wuh  [2.],  "Apples  and 
oranges  are  not  circles."  Furthermore,  [3]  (". , .  not  all  round  things  are  circles") 
contradicts  [1]  ("Things  that  are  round  are  circular.") 
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The  basic  problem  with  this  lesson  is  a  conceptual  one:  an  accurate  and  useful 
definition  of  circle  has  not  been  provided.  This  leads  to  a  failure  to  illustrate  in  any 
coherent  and  effective  way  the  differences  between  two-dimensional  and  three-dimensional 
terms,  as  well  as  to  confusion  of  terms  such  as  "circle,"  "sphere,"  "circular,"  and  "round." 

Problem  area  5.  The  lesson  lacks  an  adequate  explanation  of  context,  purpose,  or 
rationale  underlying  the  skill  objective.  Although  the  skill  objective  of  each  lesson  is 
stated  in  a  box  that  appears  on  the  title  page,  some  lessons  lack  an  introduction  stating 
why  a  skill  is  being  taught,  how  it  relates  to  other  skills,  what  uses  it  may  have  for  the 
soldier,  or  what  basic  concepts  or  definitions  are  involved  in  learning  the  skill.  In  some 
cases,  lessons  plunge  right  into  a  list  of  steps  or  rules  to  be  followed. 

Lesson  14f,  "Multiplying  and  Dividing  Fractions."  This  lesson  exemplifies  this  lack  of 
explanatory  context.  The  lesson  begins  with  a  rule: 

"To  multiply  a  fraction  by  a  fraction,  (1)  multiply  the  numerators  (upper 

numbers)  across;  (2)  multiply  the  denominators  (lower  numbers)  across." 

There  should  be  at  least  some  explanation  of  what  happens  to  fractions  when  you 
multiply  or  divide  them  (as  opposed  to  whole  numbers,  for  example),  what  occasions  or 
situations  might  require  this  skill,  and  some  explanation  of  why  this  rule  works  or  where 
it  comes  from.  This  is  lacking  not  only  in  the  short  lesson  (where  its  absence  can  only 
partially  be  justified  on  the  basis  of  a  focus  on  remediation),  but  also  in  the  long  lesson, 
where  it  cannot. 

In  the  long  lesson  (page  12,  for  example)  students  are  given  a  list  of  steps  to  follow 
for  reducing  a  fraction:  "Here  are  the  steps  for  reducing  a  fraction."  This  does  not 
provide  enough  instructional  context:  an  idea  of  what  one  wants  and  why  is  important  to 
an  understanding  of  the  process.  As  it  stands,  soldiers  are  told  to  follow  steps  blindly 
without  a  clear  understanding  why  they  are  there  in  the  first  place. 

Lesson  28b,  "Finding  Information."  The  objective  of  this  lesson  is  to  teach  students  to 
read  charts  and  tables  to  locate  information.  However,  tl.e  lesson  does  not  begin  with  a 
description  or  an  illustration  of  a  chart  or  table,  but  simply  with  instruction  about  "How 
to  read  a  row."  Unless  students  are  first  shown  how  rows  are  essential  parts  of  a  chart 
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or  table  and  how  they  are  used  to  represent  information,  teaching  them  how  to  read  a 
row  does  not  make  sense.  It  would  be  much  better  to  begin  the  lesson  with  an 
illustration,  perhaps  with  parts  such  as  rows  or  columns  labelled,  so  that  students  could 
see  the  whole  picture  and  associate  the  instruction  with  past  experience  and  with  some 
conceptual  model  of  how  parts  fit  together,  and  why  they  are  important. 


they  are  placed.  This  particular  problem  is  easier  to  detect  in  paper-based  lessons  than 
in  on-line  lessons,  primarily  because  the  whole  paper-based  lesson  is  available  to  the 
reader  at  once,  facilitating  comparison  of  different  sections.  The  examples  that  follow 
have  accordingly  been  drawn  from  a  paper-based  lesson. 


Lesson  33a,  "Record  Essential  Information,"  begins  with  a  very  good  introduction  to 
notetaking  which  clearly  states  why  knowing  how  to  take  notes  is  useful.  However,  in 
the  following  sections  basic  ideas  become  muddled  by  very  confusing  topic  organization. 

The  first  half  of  the  lesson  is  devoted  to  "informal"  notetaking.  According  to  the 
information  given  here,  informal  notetaking  has  two  forms  --  written  and  mental.  The 
first  key  point  is  the  category  of  notes  described  as  "mental  notes."  "Lists,"  (page  10)  is 
the  second  category;  however,  lists  as  they  are  described  here  are  stimuli  for  mental 
notes.  The  third  key  point  is  entitled  "Kinds  of  notetaking"  -  which  does  not  seem  to  fit 
in  here,  given  that  the  lesson  has  already  been  describing  kinds  of  mental  notetaking. 

This  category  is  subtitled,  "Written  notes."  The  next  category  is  "Time  and  reason."  This 
does  not  at  all  describe  a  kind  of  notetaking;  it  merely  explains  that  the  time  available 
and  the  reason  for  taking  notes  influence  the  type  of  notes  one  takes.  Then  the  lesson 
moves  back  to  describe  a  kind  of  written  noietaking,  "notes  for  yourself."  [Key  point  3, 
page  11.]  However,  page  12  brings  up  another  key  point  labelled  "notes  for  yourself,"  this 
time  combined  with  the  subtitle  "personal  shorthand."  The  topic  organization  in  this 
section  is  very  confusing:  key  points,  categories  and  sub-categories  have  no  coherent 
relationship.  The  reader  does  not  have  a  clear  idea  of  the  main  idea  nor  of  categories  of 
notes  when  reading  this  section. 

The  next  key  point  "Notes  for  others"  seems  out  of  place,  since  it  is  the  entire 
second  section  of  the  lesson  that  is  devoted  to  notes  for  others  (which  are  presumably 
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"formal  notes.")  Moreover,  in  the  middle  of  this  misplaced  "notes  for  others"  section,  the 
practice  item  (1.7)  asks  students  to  take  down  personal  notes  ("notes  for  yourself);  it  is 
obviously  out  of  place.  The  rest  of  the  material  on  "notes  for  others"  belongs  in  section 
II,  not  section  I. 

The  confusion  becomes  more  apparent  when  we  find  the  section  on  "taking  notes 
from  print"  of  the  first  half  of  the  lesson  on  informal  notes  (p.  14)  repeated  in  section  2 
(p.  18)  on  "formal  notes."  How  is  the  student  to  tell  whether  taking  notes  from  print  is 
classified  as  formal  or  informal;  or,  indeed,  what  the  conceptual  difference  in  fact  is 
between  formal  and  informal  notes?  This  is  quite  confusing,  especially  since  in  the  formal 
notes  section  the  lesson  talks  about  taking  notes  from  manuals  that  are  only  for  personal 
use,  such  as  the  "informal"  personal  notes  described  earlier.  In  the  end,  the  lesson  never 
explains  what  difference  it  makes  whether  the  notes  are  classified  as  formal  or  informal. 

Indeed,  the  main  point  of  the  lesson  -  developing  skill  in  noietaking,  which  is 
predicated  on  an  ability  to  distinguish  essential  from  non-essential  information  -  is  not 
taught  at  all  (problem  area  1).  At  each  point,  soldiers  are  merely  told  to  "decide  which 
information  is  important"  -  but  no  guidelines  are  given  for  how  to  make  such  a  decision, 
with  the  possible  exception  of  the  "SALUTE"  mnemonic  (S  =  subjects,  A  =  activity,  L  = 
location,  etc.).  Conventions  and  categories  for  notetaking  replace  a  focus  on  developing 
skill  in  abstracting  and  evaluating  information. 

Problem  area  7.  The  lesson  makes  abrupt  conceptual  jumps  or  relies  upon  spurious 
comparisons  or  inappropriate  juxtapositions  that  confuse  one  skill  with  another.  In  some 
cases,  skill  confusion  occurs  when  practice  or  test  questions  intended  to  test  performance 
of  one  skill  -  for  example,  notetaking  —  in  fact  require  a  student  to  use  another  skill, 
such  as  reading.  In  other  cases,  one  concept  (and  related  skill)  is  used  to  teach  students 
another  concept  (and  related  skill),  yet  the  two  arc  not  comparable. 

Lesson  Ic,  "Ordering  Numbcrs-Intervals."  This  lesson  uses  one  concept  in  order  to  teach 
another,  but  these  concepts  are  fundamentally  different,  leading  to  failure  of  students  to 
master  the  intended  skill.  The  lesson  begins  with  an  illustration  of  a  thermometer  whose 
temperature  is  going  down.  This  is  a  good  way  to  illustrate  intervals;  however,  the 
linkage  between  the  "increasing"  negative  numbers  on  the  temperature  scale  and  the  idea 
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that  the  temperature  is  decreasing  is  not  made  explicit.  This  is  crucial,  as  it  seems  to  be 
the  main  point  of  the  illustration. 


The  entire  first  section  of  this  lesson  is  devoted  to  the  temperature  scale;  on  page 
seven,  however,  the  lesson  suddenly  jumps  to  practice  questions  that  have  nothing  to  do 
with  temperature  or  scale,  but  ask  the  student  to  sequence  numbers  and  decimal  quantities 
entirely  independently  of  a  scale.  The  comparison  between  temperature  scales  and  decimal 
quantities  is  inappropriate;  unless  some  of  the  similar  notions  inherent  in  interval  and 
distance  on  a  number  scale  and  the  notion  of  negative  numbers  are  made  explicit.  Thus 
the  comparison  is  of  little  value  to  the  student  in  learning  how  to  sequence  negative 
decimals. 

Moreover,  the  lesson  began  by  presenting  the  concept  of  negative  decimals  as 
temperature  marks  on  a  thermometer.  The  focus  was  upon  identifying  (reading)  negative 
temperatures.  This  task  does  not  require  a  student  to  arrange  numbers  in  a  sequence;  it 
only  requires  him  or  her  to  read  a  thermometer.  This  represents  a  spurious  equivalence 
of  tasks.  Expecting  students  to  make  the  conceptual  jump  from  reading  off  numbers  on 
an  interval  scale  to  sequencing  and  to  notions  of  quantity  phrased  in  terms  of  least  and 
greatest  is  problematic. 

The  notion  of  "negative  quantity"  upon  which  some  sections  of  the  lesson  and  test 
questions  are  based  is  also  difficult  to  visualize.  The  lesson  requires  students  to  deter¬ 
mine  which  of  a  series  of  negative  decimals  is  "larger"  or  "smaller."  These  terms  carry 
over  the  concept  of  "large"  and  "small"  from  positive  numbers  which,  when  applied  to 
negative  numbers,  leads  to  errors,  since  students  automatically  consider  the  "size"  of  the 
number  in  positive  terms.  Moreover,  the  lesson  never  makes  explicit  the  fact  that  the 
"larger"  the  negative  number  appears,  the  "smaller"  it  in  fact  is.  Instead,  the  lesson  gives 
students  this  rule: 

"To  determine  whether  a  number  comes  before,  after,  or  in  between  a  group  of 
numbers,  put  the  numbers  in  sequence  from  least  to  greatest,  then  compare  your 
new  number  to  each  of  the  sequenced  old  numbers,  one  at  a  time." 

"Determining  whether  a  number  comes  before,  after,  or  in  between"  is  exactly  what 
sequencing  involves,  so  essentially  this  rule  reduces  to  the  tautology,  "To  sequence  a 
number,  put  it  in  sequence."  The  lesson  inappropriately  uses  one  skill  (reading  tempera- 
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lures  on  a  scale)  to  "explain"  another  (negative  decimal  "quantity"),  failing  to  make 
explicit  the  relationship  between  distance  on  an  interval  number  scale  and  negative  quan¬ 
tities. 

Lesson  3a,  "Identify  Degrees  and  Mils."  The  stated  purpose  of  the  lesson  is  cloudy:  "How 
to  identify  angular  measurements  by  using  degrees  and  mils."  Does  tliis  mean  that  the 
lesson  will  teach  students  to  identify  which  measurements  are  degrees  and  which  are  mils 
“  that  is,  to  distinguish  between  measurements  in  degrees  and  mils?  Or  will  students 
learn  how  to  measure  or  compare  angles  in  both  systems?  This  purpose  becomes  even 
more  fuzzy  when  juxtaposed  with  the  "second"  purpose  of  the  lesson:  "You  will  also 
review  how  to  recognize  temperature  in  both  degrees  fahrenheit  and  degrees  centigrade." 
Juxtaposing  angular  measurements  and  temperature  measurements  as  if  they  were  somehow 
"equivalent"  is  a  false  comparison:  angular  degrees  have  little  to  do  with  temperature 
degrees,  except  for  the  fact  that  they  are  both  called  "degrees."  They  are  measured  in 
entirely  different  ways  and  they  are  applied  to  entirely  different  entities;  they  are  thus 
conceptually  quite  distinct  and  their  Juxtaposition  here  does  not  clarify  what  is  already  a 
rather  muddy  lesson  purpose. 

This  confusion  over  what  skill  the  lesson  is  in  fact  teaching  is  carried  over  to  the 
lest.  Item  4  asks  for  the  angle  of  the  soldier’s  arm  (in  salute  posture).  This  is  not  an 
identification  question  involving  mils  or  degrees;  it  tests  previous  knowledge  of  proper 
salute  posture.  Item  3  shows  an  angle  with  a  45-degree  measure.  The  question  asks 
whether  45  degrees  refers  to  temperature  or  angular  measurement.  This  does  not  test  any 
ability  to  "distinguish"  different  types  of  degree  measures  from  one  another,  but  simple 
reading  ability.  One  must  concur  with  the  soldier  who  said  that  sometimes  the  questions 
are  so  obvious  that  ". . .  they  begin  to  play  with  your  mind." 

Problem  area  8.  The  lesson  contains  internal  logical  problems:  parts  of  the  same 
lesson  contradict  each  other,  or  the  le.s.son  teaches  one  thing  but  practice  questions 
require  another. 

Lesson  41b,  "Identify  and  Use  Hand  and  Arm  Signals."  The  short  lesson  41b  draws  a  basic 
categorical  distinction  between  mounted  and  dismounted  arm  signals:  it  illustrates  both 
types,  and  teaches  that  these  are  two  separate  categories  without  overlap.  However,  the 
long  lesson  abandons  the  mounted/dismounted  distinction,  and  tells  the  student:  (p.6)  "The 
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signals  in  this  part  of  the  lesson  may  be  given  from  the  ground  or  a  vehicle," 
contradicting  the  information  presented  in  the  short  lesson.  [This  is  also  a  case  of 
short/long  lesson  content  mismatch,  problem  area  16.] 


In  lesson  28b,  section  2,  page  12,  there  is  a  description  and  illustration  of  columns: 
"On  a  printed  page,  a  column  is  made  up  of  two  or  more  vertical  sections  of  lines  of  the 
page."  [The  language  here  is  awkward  and  confusing:  what  are  "vertical  sections  of 
lines?]  Practice  questions  2.1  and  2.4  look  like  perfectly  legitimate  columns,  but  the 
answer  key  says  2.1  is  not  a  column  because  the  numbers  "...  do  not  go  up  and  down  in 
a  straight  line."  Then,  logically,  2.4  is  not  a  column  either,  because  those  numbers  "do 
not  go  up  and  down  in  a  straight  line"  either  --  in  both  cases  one  digit  "sticks  out."  Yet 
the  answer  key  says  2.4  is  a  column  because  ". . .  the  numbers  that  form  this  column  run 
up  and  down."  So  do  the  numbers  in  2.1!  Logical  inconsistencies  result  from  the 
confusing  and  misleading  verbal  descriptions  in  this  lesson. 


Presentation  or  Delivery  Problems 


The  previous  section  dealt  with  various  types  of  conceptual  problems  present  in  JSEP 
lessons.  This  section  will  consider  problems  affecting  presentation  of  lesson  material, 
including  answer  formats,  language,  practice  items,  illustrations,  and  instructional  methods. 


Problem  area  9.  The  lesson  has  mechanical  or  format  problems  with  test  or  practice 
questions  that  interfere  with  answering  the  question  in  the  correct  manner.  In  some 
cases  the  question  focuses  more  on  the  format  than  upon  the  skill  or  concept  behind  the 
question,  thus  confusing  students: 


The  01  series  on  JSEP  has  many  test  questions  that  are  confusing  to  the  students.  I 
feel  that  if  those  questions  were  worded  differently,  more  of  my  students  would  pass 
the  diagnostic  test  and  save  a  lot  of  time.  A  student  may  know  the  skill  itself,  but 
miss  the  question  because  he  didn’t  know  what  was  being  asked.  [Instructor’s  p- 
note,  2/5/88] 


In  one  sense  this  particular  problem  could  be  seen  as  related  to  a  failure  of  some 
lessons  to  teach  the  concepts  underlying  skills  in  ways  that  focus  students’  attention  on 
the  concepts  themselves  rather  than  upon  aspects  of  format  or  mechanics.  Questions 


worded  so  that  the  main  skill  objective  is  not  clear  may  be  those  in  which  other  aspects 
of  the  problem  --  such  as  format  -  take  precedence.  On  the  JSEP  attitude  survey, 
students  said:  "Sometimes  the  computer  doesn’t  explain  how  to  answer  some  of  its 
questions."  "At  times  the  explanations  were  unclear  as  for  their  answers."  "Sometimes  I 
would  get  so  frustrated  I  would  put  wrong  answers  because  of  abbreviations."  Format 
problems  thus  can  not  only  obscure  the  skill  being  tested,  but  also  lead  to  frustration  that 
can  affect  student  performance. 

In  Lesson  lb,  "Writing  Sequential  Numbers,”  it  appears  that  mechanical  errors  can  be  a 
major  source  of  wrong  answers,  even  when  the  student  knows  the  answer.  Often,  it  is 
not  clear  what  form  an  answer  should  be  in:  For  example,  in  test  item  3,  the  question 
does  not  specify  if  spaces  (space  bar)  should  be  used,  or  if  so,  how  many  spaces  should  be 
inserted  between  numbers.  Nor  does  it  state  if  and  where  commas  or  periods  should  be 
used.  Moreover,  all  the  items  on  the  practice  section  have  been  sequenced  low  to  high, 
but  test  item  6  asks  for  a  high  to  low  sequence.  This  sudden  change  can  be  misleading. 
These  mechanical  difficulties  led  one  student  to  complain,  "Either  the  scoring  [on  this 
lesson]  is  screwed  up,  or  the  instructions  are  so  confusing  I  don’t  know  what  they  want." 
[Personal  note] 

Lesson  34d,  "Outlining,"  poses  major  mechanical  difficulties  for  students,  since  answers  to 
practice  and  test  questions  depend  on  repeatedly  pressing  the  space  bar  the  right  number 
of  times  in  order  to  get  the  code  elements  in  the  correct  places.  According  to  JSEP  staff 
at  one  site,  the  lesson  teaches  students  more  about  the  mechanics  of  manipulating  the 
computer  than  it  does  about  outlining. 

Similar  problems  plague  Lesson  6b,  "Alining  Parts  and  Equipment"  Test  items  require  the 
student  to  use  arrows  to  manipulate  objects  on  the  screen,  but  in  some  cases  it  is  very 
difficult  to  tell  if  one  has  moved  the  given  object  far  enough  or  if  one  has  gone  too  far. 

In  test  item  13,  for  example,  it  is  exceedingly  difficult  to  tell  exactly  where  the  45  degree 
position  is.  Thus,  even  though  the  student  may  understand  the  concept  and  know  what  he 
or  she  must  do,  the  mechanics  of  answering  the  questions  can  defeat  their  purpose. 

Problem  area  10.  The  lesson  has  language  problems  such  as  inconsistent  terminology, 
ambiguities,  awkward  or  confusing  phrasing,  grammatical  errors,  or  typographical  errors. 
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(This  section  will  be  devoted  to  problems  in  the  lessons  themselves;  for  language  problems 
affecting  test  items,  please  see  problem  area  18.) 

Language  problems  were  common  in  the  JSEP  lessons  reviewed.  Ranging  from  minor 
typographical  errors  to  ambiguous  and  confusing  explanations,  they  became  particularly 
troublesome  when  they  affected  presentation  of  basic  concepts.  Examples  follow. 

Lesson  9a:  "Identifying  Angles."  (p.2)  "Two  lines  segments  emerging  from  a  common 
endpoint  make  an  angle."  It  is  not  clear  whether  the  writer  meant  "lines"  or  "line 
segments."  The  verb  phrase  "emerging  from  a  common  endpoint"  is  awkward;  things 
normally  emerge  from  beginnings,  not  endpoints. 

(p.  18)  The  angle  of  site  forms  with  the  gun  or  battery  as  the  vertex.  One 
line  segment  is  the  gun  to  the  target  line.  The  other  segment  is  the  gun  to 
the  ground  line,  also  called  the  base  line. 

This  is  confusingly  worded,  and  could  be  written  much  more  clearly  (if  I  have  understood 
it)  as; 


The  vertex  of  the  angle  of  site  is  the  gun  or  the  battery.  One  line  segment  extends 
from  the  gun  to  the  target.  The  other  line  segment  runs  from  the  gun  along  the 
ground. 


Lesson  5c:  "Read  and  Interpret  a  Gage."  Inconsistent  terminology  in  different  parts  of 
this  lesson  can  cause  students  difficulties.  The  short  lesson  tells  students  they  need  to 
perform  two  steps  in  order  to  read  multimeters.  The  first  is  to  "select  the  correct  band." 
The  short  lesson  uses  the  term  "band"  consistently  and  offers  practice  in  identifying 
bands.  However,  the  long  lesson  states  the  rules  a  little  differently:  students  need  to 
"select  the  correct  scale."  In  the  long  lesson,  the  focus  is  upon  "scale."  "Scale"  and 
"band"  are  not  the  same:  this  distinction  needs  clarification  and  illustration. 

Lesson  12h:  "Estimating  a  Sum  or  Difference."  In  this  lesson  on  estimating  sums  or 
differences,  a  prepositional  error  changes  the  meaning  of  a  basic  rule:  (page  4) 

"Determine  which  two  tens  the  number  you  are  rounding  to  is  between."  (639 
is  between  630  and  640) 
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"To"  is  an  error;  the  sentence  should  read,  "Determine  which  two  tens  the  number  you  are 
rounding  is  between."  The  number  being  rounded  and  the  number  being  rounded  iq  are 
quite  different. 

Lesson  29b:  "Use  a  Key,  Legend,  or  Parts  List"  This  lesson  instructs  students  in 
reading  keys,  legends,  and  parts  lists.  The  short  lesson  (pages  16-17)  gives  rules  for 
using  parts  lists  as  follows: 

"A  parts  list  gives  the  names  and  numbers  for  the  parts  shown  in  the 

illustration .... 

To  find  a  part  on  an  illustration: 

•  Find  the  name  on  the  parts  list 

•  Look  for  the  part  number 

•  Look  for  the  part  number  on  the  illustration 

In  this  case  the  term  "part  number"  is  ambiguous,  for  there  are  two  kinds  of  "part 
numbers"  --  one  relating  to  the  sequence  of  parts  found  in  the  illustration,  and  one  that 
serves  as  the  part  identification  number.  These  are  not  distinguished  from  each  other. 


Lesson  28b:  "Finding  Information."  There  are  numerous  examples  of  awkward  and 
ambiguous  wording  in  the  paper-based  long  lesson. 


p.  12.  "On  a  printed  page,  a  column  is  made  up  of  two  or  more  vertical 
sections  of  lines  on  the  page." 

p.  24.  "You  will  find  the  row  heading  from  the  intersection  of  the  row  and 
column." 

p.  25.  "Sometimes  several  rows  of  information  about  one  item  will  be  listed 
below  its  first  listing,  and  the  information  that  identifies  it  will  not  be 
repeated." 


Lesson  11a:  "Identify  Shape  and  Position  Terms."  There  are  similar  problems  in  this 
lesson. 


I 
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p.  4.  You  will  often  need  to  draw  a  circle  when  instructed  to  "draw  a  circle..." 
p.  2.  Some  endings  to  words  change  their  meanings  a  little  bit,  like  -ly  and 
-ed. 

p.  5.  Things  that  are  round  are  circular,  or  are  shaped  like  spheres,  circles, 
globes,  or  balls.  In  everyday  usage,  the  word  circle  refers  to  a  general 
shape.  An  object  does  not  need  to  be  a  perfect  sphere  to  be  called  round. 
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[Paragraph  is  confused,  changes  topic  from  "round"  to  "circle"  in  second 
sentence,  :hen  goes  back  to  "round";  also,  redundant:  "circular  things  are 
shaped  like  circles."] 

Problem  area  11.  Practices  or  examples  are  poorly  chosen,  inappropriate  or  confus¬ 
ing.  given  the  aim  of  the  les.son.  In  some  cases,  practice  items  do  not  give  the  soldier 
practice  in  the  skills  objective  of  the  lesson,  but  in  peripheral  skill  areas.  Thus,  they 
misrepresent  the  skill  objective  of  the  lesson. 

Lesson  27c:  "Finding  Information  in  a  Book."  The  skill  objective  of  this  lesson  is  to 
teach  students  how  to  locate  specific  types  of  information  using  features  such  as  an 
index,  appendix,  table  of  contents,  or  glossary.  The  focus  should  be  on  giving  students 
practice  in  how  to  determine  what  kind  of  information  is  found  in  each,  and  how  to  make 
judgments  as  to  where  a  particular  kind  of  information  would  be  found.  However,  the 
practice  items  fail  to  target  this  essential  skill;  instead,  they  repeatedly  ask  the  student 
to  locate  page  numbers  for  specific  given  topics.  Most  of  the  items  are  of  the  format: 

"On  what  page  would  you  find  the  appendix  concerned  with  the  vocational 
training  institute?" 


This  type  of  item  does  not  give  the  student  practice  in  deciding  where  different 
kinds  of  information  might  be  found.  It  does  not  illustrate  how  an  index,  for  example, 
might  be  more  appropriate  to  use  than  a  table  of  contents,  depending  on  the  type  and 
degree  of  specificity  of  the  required  information.  Asking  students  to  read  off  page 
numbers  in  samples  of  appendices,  tables  of  contents,  and  so  forth,  only  gives  practice  in 
peripheral  table  or  list  reading  skills,  not  in  the  intended  skill  objective  of  the  lesson. 


Lesson  11a:  "Identify  Shape  and  Position  Terms."  Section  1,  page  2,  contains  a  very 
confusing  example.  The  intention  is  to  illustrate  how  "suffixes"  alter  the  meaning  of 
words  --  in  this  case,  the  suffixes  -ly  and  -cd. 


aline 

alined 

horizontal 

horizontally 


do  now 
already  done 
a  position 

a  descriptive  direction 
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As  it  stands  this  example  does  not  illustrate  the  meanings  of  -ed  or  -ly;  it  merely 
confuses  the  reader  by  seeming  to  provide  definitions  for  the  terms:  e.g.,  "Aline  means 
‘do  now.’"  Contrasting  "a  position"  and  "a  descriptive  direction"  does  not  illustrate  the 
meaning  of  -ly  very  effectively. 

Lesson  Ig:  "Rounding  Numbers."  The  first  practice  question  in  rounding  on  page  10  asks 
the  soldier  to  round  a  number  ending  in  5.  The  task  involved  here  (rounding  to  the 
nearest  100)  has  not  been  explained  yet;  the  next  section  of  the  lesson  deals  with 
rounding  to  the  nearest  5,  and  the  "rules"  for  rounding  to  the  nearest  100  do  not  come 
until  later  in  the  lesson. 

On  page  20,  we  find  an  example  of  a  practice  question  that  requires  the  student  to 
"round"  a  number  for  which  rounding  is  in  fact  not  necessary.  The  feedback  to  this 
question  even  states  that  it  is  "tricky"  -  if  so,  then  it  should  not  appear  as  a  practice 
item. 

Lesson  Id:  "Ordering  Numbers  -  Greatest  and  Least."  The  objective  of  this  lesson  is  to 
teach  the  skill  of  ordering  numbers,  or  recognizing  relative  values.  The  walk-through 
example  illustrates  how  to  compare  numbers  by  lining  them  up  in  place  value  columns.  All 
the  numbers  to  be  compared  have  the  same  number  of  digits  (or  places.)  However,  the 
practice  question  on  page  7  suddenly  asks  students  to  compare  numbers  in  which  the 
number  of  places  is  not  the  same;  moreover,  the  same  number  (22871)  occurs  twice  in  the 
list.  No  instruction  has  been  given  as  to  what  to  do  when  the  numbers  do  not  have  the 
same  number  of  digits.  This  practice  is  thus  inappropriate. 

In  the  section  on  fractions  (page  17)  the  practice  item  on  page  24  asks  students  to 
compare  fractions  where  both  the  numerator  and  denominator  are  different  before  they 
get  to  the  section  of  the  lesson  that  in  fact  explains  how  to  do  this.  This  practice  item 
is  out  of  place. 

Problem  area  12.  Feedback  to  practice  question  answers  is  inappropriate.  In  some 
cases,  the  feedback  incorrectly  "presumes"  the  user  made  a  particular  kind  of  error  in 
answering  the  question.  In  other  cases,  the  feedback  is  contradictory  or  irrelevant  to  the 
intent  of  the  question. 
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Lessoo  32c:  "Locating  Infonnation  on  a  Fonn."  (Long  lesson,  section  B,  page  17)  The 
question  presents  a  section  of  a  form,  and  asks  the  soldier  to  locate  infonnation  from  a 
block  on  the  form. 

[Question)  How  was  the  failure  first  detected?  lype  in  the  conect  answer. 
Answer)  >  normal  cm 

Feedback)  No,  the  failure  was  detected  during  normal  op  (eration). 

This  is  the  sort  of  error  that  prompted  a  soldier  to  comment  at  one  point  in  a  lesson, 
"Hey,  your  answer  and  my  answer  are  the  same.  How  can  I  be  wrong?"  [Personal  note] 

Lesson  4a:  "24-Hour  Time."  (page  9)  The  answer  to  a  practice  question  requires  the 
student  to  use  12  hour  time  notation.  A  typing  error  in  "PM"  elicits  this  response: 

No,  the  rule  you  should  use  involves  subtraction.  Did  you  subtract  correctly? 

Yes,  I  did,  but  that  was  not  the  problem.  Although  feedback  is  designed  to  be  as  helpful 
as  possible  to  the  student,  incorrect  presumptions  such  as  this  can  be  misleading  - 
especially  if  the  student  is  not  aware  that  he  has  made  a  mechanical  error  in  the  answer. 
Such  feedback  does  not  provide  adequate  information  about  the  required  answer  format. 

Lesson  29c  "Use  A  Gross-Section."  Page  four  illustrates  two  flame  devices,  one  aoss- 
section  with  its  parts  labelled  and  one  non-cross  section  without  labels.  The  question 
asks  the  soldier  to  "toudt  the  rim  on  the  flame  device."  If  the  student  touches  the  rim 
that  is  labelled,  [a  logical  and  defensible  response,  given  that  the  aim  of  the  lesson  is  to 
give  students  practice  with  cross-sectional  illustrations]  the  response  is: 

No.  This  is  the  cross-section.  Touch  the  rim  on  the  other  bottle. 

This  feedback  is  not  helpful  to  the  student,  for  it  does  not  specify  why  this  response  is 
wrong.  Moreover,  the  peculiar  boldfaced  "is”  does  not  make  sense.  It  implies  that  the 
question  asked  was  something  like,  "Is  this  or  is  this  not  the  cross-section?" 

Furthermore,  the  point  of  this  question  is  not  dear.  If  it  is  to  get  the  soldier  to 
distinguish  between  cross-section  and  illustration,  then  the  question  should  specify  this. 
"Flame  Device"  alone  is  ambiguous. 
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Problem  area  13.  Lesson  graphics,  illustrations,  or  captions  are  confusing,  mis¬ 
labelled.  or  misdrawn.  (This  section  considers  only  graphics  or  illustrations  in  the  lessons 
themselves,  not  in  the  tests.) 

Lesson  3a:  "Identify  Degrees  and  Mils." 


Ns>i/V  y-ou  tr-y  j-t. 

M-eaL»ciT“«  ^■‘sv  i  a.'t  i  s-n  f  7~ofn  "t  knoum 

point  to  th-5  tar-g^t. 


I  S>=CK  to  n^v  i  .Bijj  I 


This  diagram  on  page  20  (short  lesson)  is  misleading.  Reference  point  A  appears  to  be 
almost  directly  above  target  B;  only  a  small  arrow  points  to  a  different  point  farther  out. 
I  originally  saw  only  the  A,  which  led  to  a  wrong  answer. 
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On  page  32  there  is  a  rather  confusing  illustration.  The  diagram  can  be  interpreted 
as  showing  two  separate  thermometers  (with  the  Fahrenheit  on  the  left  and  the  centigrade 
on  the  right')  or  as  showing  one  thermometer  with  two  separate  scales  (the  latter  probably 
being  the  intended  interpretation). 


Rll  right,  what  is  tha 
tamp-erature  ir  dagraas 
Cant  i gr ada  in  t ha 
t harmomatar  at  tha  left? 


Typa  in  tha  cor-ract 
numt>ar .  26  C 

You ’ ra  right,  26*C. 


i&RCK  ro  raviawl  I  NEXT  to  contiTTual 


The  question  asked  is: 

What  is  the  temperature  in  degrees  centigrade  in  the  thermometer  at  the  left? 
This  question  adds  to  the  ambiguity,  implying  either  (1)  that  the  student  should  simply 
read  the  one  thermometer  (whose  position  is  left  of  the  text  on  the  screen)  or  (2)  that 
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the  student  should  read  the  "thermometer"  on  the  left  (whose  scale  is  in  degrees  Faren- 
heit)  and  then  "convert"  the  reading  to  centrigrade  by  reading  across  to  the  other  scale. 

A  clearer  illustration  would  be  helpful  here.  (I  made  the  mistake  of  seeing  the  diagram  as 
two  separate  thermometers.) 

Lesson  27c:  "Finding  Information  in  a  Book."  (Long  lesson.  Section  D)  In  the  illustra¬ 
tion  requiring  the  student  to  touch  the  term  "center  point,"  there  is  more  than  one 
"center  point"  term  printed  on  the  screen.  I  spent  five  minutes  touching  one  of  the 
"center  point"  terms  on  the  screen,  without  realizing  that  there  were  others.  The 
computer  would  not  respond.  Finally  I  realized  there  was  another  "center  point"  term  on 
a  different  part  of  the  screen,  which  was  the  one  that  had  to  be  touched  in  order  to  get 
the  computer  to  respond.  This  was  not  specified  in  the  instructions. 

Lesson  14fc  "Multiplying  and  Dividing  Fractions."  (Long  lesson,  page  5) 


Scm'et  i  .Ti-es  you  multiply  mom-a  tha.n  two  f r'act :  or;s .  /  h-e 

sarn-e  1  ap>c  1  y . 
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The  indicator  lines  in  this  illustration  of  fractions  are  confusing.  They  seem  to  show  that 
somehow  the  denominators  4  and  6,  when  multiplied  together,  become  the  numerator  of 
the  fraction  beneath.  [The  same  thing  holds  for  3x7  and  8.] 

Lesson  12h:  "Estimating  a  Sum  or  Difference."  Pages  3  and  4  show  number  lines  in  order 
to  illustrate  the  concept  of  rounding.  However,  the  "midpoint"  on  the  number  line 
connecting  two  numbers  is  not  placed  in  the  middle  (where  it  should  be)  but  slightly  to 
the  side. 


sajT^-e  wfi'ert  r'c-unci  i  ri.-=;  -to  p  is.ee 


Rc-und  6  3’?  "to  the  near^est  ter.. 

•  'Determine  which  two  tens  t.he  rAJLmis-er  ycu. '  r« 
rounding  to  is  between. 

('■•■3?  is  fc-etween  6  30  a.nd  6  4£0 

•  rrsk  vouirsel  f  which  ra_une-er  6  39  is  closer  tc 


639 

' 


630 


©35 
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It‘s  closer  to  640.  So... 

639  rounded  to  the  nearest  ten  is  640. 


!~RCK  to  review 


INEIXT  to  continuel 


Especially  since  the  point  is  to  estimate  how  far  a  number  is  from  another,  these 
distances  should  be  represented  as  accurately  as  possible.* 
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Problem  area  14.  The  lesson  uses  an  inappropriate  instructional  method:  for 
example,  "telling"  rather  than  teaching,  or  using  a  practice  item  prematurely  that  "traps" 
students  into  a  wrong  answer. 

Lesson  4a:  "24-Hour  Time."  The  long  lesson  section  on  adding  military  time  begins  by 
offering  students  a  very  useful  option:  if  they  would  like  to  see  more  "information"  on 
the  topic  they  may  choose  "I,"  or,  if  they  feel  they  already  know  the  material,  they  may 
choose  "P"  and  go  directly  to  more  "practice."  The  lesson  says: 

If  you’re  sure  you  know  this  information,  touch  the  "P"  box  below  to  go 
directly  to  the  practice.  If  you  aren’t  sure  you  know  this  information,  touch 
the  "I"  box  below. 

Touching  the  "P"  box  does  indeed  bring  up  practice  questions.  However,  touching  the  "I" 
box  also  brings  up  a  practice  question,  not  information: 

(p.  34)  The  time  is  now  1548.  The  enemy  attaek  began  3  hours  and  15  minutes 
ago.  At  what  time  did  the  enemy  attack  begin?  T^pe  in  the  correct  answer." 

This  is  almost  as  if  the  intent  of  the  program  were  to  prove  to  the  student  that  he  or 
she  in  fact  did  not  know  the  answer.  This  is  inappropriate  as  an  instructional  method 
given  that  the  student  has  specifically  requested  more  information,  not  practice,  because 
he  or  she  docs  not  know  the  material.  This  practice  question  appears  designed  to  "trap" 
the  student  into  a  wrong  answer,  or  to  "rub  in"  the  fact  that  he  or  she  does  not  know 
the  material. 

Lesson  12c:  "Add  or  Subtract  Military  Time."  A  similar  problem  occurs  here.  The  lesson 
lacks  any  sort  of  introduction  or  statement  of  context.  It  immediately  presents  students 
with  a  test  question: 

(p.  1)  Your  unit  was  participating  in  NBC  training.  After  the  alarm  sounded, 
you  were  told  to  wear  your  mask  and  hood  for  exactly  one  hour  and  forty 
minutes.  The  alarm  sounded  at  1052  hours.  What  time  was  it  when  you  took 
off  your  mask  and  hood?  Type  your  answer. 
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This  seems  designed  only  to  trap  students  into  making  a  mistake,  to  show  them  they  do 
not  know  how  to  solve  the  problem.  The  instructional  value  is  negative.  Page  two  jumps 
immediately  to  statement  of  the  "rule"  used  to  solve  the  problem. 

In  the  long  lesson,  the  problem  seen  in  lesson  4a  reappears.  Students  are  given  a 
choice  to  select  the  I  (information)  option  or  the  P  (practice)  option.  Selecting  informa¬ 
tion  again  brings  up  a  test-type  question  that  imitates  the  format  found  at  the  beginning 
of  the  short  lesson.  Students  who  select  information  do  so  presumably  because  they  do 
not  want  to  confront  a  practice  question.  This  is  an  inappropriate  instructional  technique. 

Lesson  Id:  "Ordering  Numbers  —  Greatest  and  Least"  This  lesson  contains  an  example 
of  teaching  students  to  use  a  method  to  solve  a  problem  that  results  in  a  wrong  answer. 

In  the  long  lesson,  page  56,  students  are  taught  a  way  to  find  a  common  denominator  by 
multiplying  denominators  of  fractions.  The  example  given  produces  an  answer  of  40. 
However,  the  next  page  of  the  lesson  tells  students:  "40  is  not  the  lowest  common 
denominator,"  and  that  they  must  find  the  smallest  one  possible.  However,  no  further 
instruction  is  given,  nor  is  the  correct  answer  to  the  problem  given.  This  is  highly 
inappropriate.  If  the  method  taught  gives  40  as  the  answer,  and  40  is  not  correct,  then 
why  teach  this  method?  Either  additional  instruction  in  how  to  use  this  technique  is 
needed,  or  it  should  not  be  taught  at  all. 

Lesson  41b:  "Identify  and  Use  Hand  and  Arm  Signals."  The  long  lesson  asks  students  if 
they  remember  the  meaning  of  a  particular  signal: 

(p.  7)  Do  you  know  what  this  signal  means?  [Yes  or  No] 

If  the  student  answers  Yes  the  program  responds. 

Good.  Then  the  first  signal  will  be  easy. 

The  lesson  proceeds  to  explain  the  meaning  of  that  same  signal. 

If  the  student  already  knows  what  it  means,  why  teach  it?  This  wastes  student  time  and 
is  needlessly  repetitive.  There  should  be  some  way  to  opt  out  of  unnecessary  instruction. 


especially  in  this  case  in  which  the  student  is  led  to  expect  such  a  choice  by  the  very 
fact  that  the  program  asks  if  he  or  she  already  knows  the  material. 

The  problem  recurs  on  page  19  in  the  practice  section.  The  lesson  illustrates  signals 
and  asks. 

Do  you  already  know  any  of  these?  Touch  Yes  or  No. 

Touching  Yes  elicits  the  response. 

Fine!  You’re  well  on  you  way  to  mastering  all  7  signals.  Your  first  signal  is 
next.... 


The  lesson  then  proceeds  to  teach  all  seven  signals,  some  or  all  of  which  the  student 
already  knows.  Asking  the  question  leads  the  student  to  expect  an  option  that  is  then 
denied,  resulting  in  needless  repetition. 


Problems  In  Short/Lx»ng  Lesson  Compatibility  and  Tests 

Problem  area  15.  The  short  and  the  long  lessons  use  a  similar  or  identical  approach 
to  the  lesson  material.  The  basic  problem  here  from  the  point  of  view  of  instructional 
excellence  is:  If  students  have  trouble  with  the  way  material  is  presented  in  the  short 
lesson,  will  more  of  the  same  approach  in  the  long  lesson  be  any  better?  Despite  the 
intention  of  JSEP  lesson  design  to  provide  review  in  the  short  lesson  and  instruction  in 
the  long  lesson,  this  contrast  is  in  reality  often  obscured  by  repeating  short  lesson 
material  in  the  long  lesson,  or  by  failing  to  take  an  approach  that  is  sufficiently  different 
in  the  long  lesson  to  qualify  as  "instruction"  (as  opposed  to  review.)  In  fact,  the 
review/instruction  distinction  is  an  artificial  one  for  many  lessons. 

In  such  lessons,  whole  segments  of  the  short  lesson  sometimes  reappear  in  the  long 
lesson.  In  other  cases  the  long  lesson  presents  material  in  essentially  the  same  manner  as 
the  short  lesson,  though  with  some  additional  remarks  or  explanations.  Yet  overall  these 
additional  explanations  and  practice  items  do  not  significantly  differ  from  the  short  lesson 
explanations  and  practice  in  their  focus  or  approach.  Examples  follow. 
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Lesson  Ig:  "Rounding  Numbers."  The  short  lesson  presented  the  rules  governing  each 
case  of  rounding,  based  on  the  particular  place  one  is  rounding  to.  Section  A,  page  1  of 
the  long  lesson  introduces  the  topic  of  rounding  using  exactly  the  same  illustration  as  in 
the  short  lesson,  but  there  is  an  error  in  the  caption.  On  page  2,  "instruction"  consists 
of  telling  the  soldier  that  "27  rounded  to  the  nearest  10  would  be  30.  (27  is  closer  to  30 
than  to  20.)"  Simply  telling  the  student  what  the  answer  is  without  any  walk-though  or 
explanation  is  not  instruction.  Page  4  introduces  a  section  on  whole  numbers  which  is 
taken  directly  from  the  diagnostic  lesson.  Page  10  presents  an  "overview  of  rules"  that 
appears  to  be  a  little  more  general  than  the  diagnostic  lesson,  since  a  section  is  labeled 
"rounding  to  a  specific  place  value."  But  the  lesson  then  immediately  goes  back  to 
repeating  lists  of  specific  rules  for  each  case  of  rounding  to  a  different  place  value,  just 
as  was  done  in  the  diagnostic  lesson.  The  long  lesson  in  fact  does  not  provide  any  new 
instruction  in  rounding;  it  is  based  entirely  on  the  approach  taken  in  the  short  lesson. 

Lesson  3a:  "Identify  Degrees  and  Mils."  The  long  lesson  also  repeats  material  from  the 
short  lesson,  but  adds  some  new  contextual  information  such  as  why  knowing  about  mils 
and  degrees  is  important.  The  discussion  beginning  on  page  4,  however,  introduces 
material  that  does  not  really  serve  the  instructional  purpose  of  the  lesson.  It  goes  back 
to  a  very  basic  level,  describing  in  detail  what  line  segments,  rays,  and  angles  are.  This 
is  not  the  focus  of  the  lesson;  it  is  "instruction"  in  peripheral  areas.  Section  B  repeats 
the  same  material  from  the  short  lesson,  but  appends  a  game  ("tank  shoot")  which  is  wise, 
helping  to  relieve  what  may  be  monotonous  repetition.  Section  C  also  repeats  much  of 
the  same  material  found  in  the  short  lesson.  Though  the  long  lesson  does  contain  some 
new  material,  the  approach  is  essentially  the  same  as  that  of  the  short  lesson  with  the 
exception  of  the  game.  (The  game,  however,  qualifies  as  practice,  not  instruction.) 

Problem  area  16.  The  long  lesson  and  short  lesson  do  not  match  each  other  in 
content:  one  lesson  teaches  something  the  other  doesn’t.  In  contrast  to  the  above 
described  problem  where  both  lessons  take  the  same  approach  to  the  lesson  material,  in 
some  cases  the  lessons  differ  from  each  other  in  content.  In  fact,  from  an  instructional 
point  of  view  it  should  be  the  other  way  around:  Lessons  should  differ  in  approach  but 
match  in  content. 
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Lesson  27c:  "Scanning  for  Facts."  The  long  lesson  contains  a  section  devoted  to 
"skimming"  while  the  short  lesson  is  concerned  entirely  with  scanning.  In  fact,  the  long 
lesson  points  out  the  difference  between  scanning  and  skimming,  and  then  confuses  them 
later  in  the  practice  questions: 

(p.  42)  Directions.  Skim  the  paragraph  for  the  main  idea. 

(paragraph  here) 

What  was  the  paragraph  you  just  scanned  about? 

This  is  a  case  where  content  mismatch  should  not  have  much  impact  on  student  perfor¬ 
mance  since  the  extra  content  is  not  tested.  It  merely  becomes  a  source  of  confusion 
within  the  lesson  practice  questions. 

Lesson  15d:  "How  to  Use  a  Protractor."  The  short  and  long  lessons  for  this  prerequisite 
competency  are  essentially  very  different  in  the  skills  that  they  teach.  The  short  lesson 
begins  with  a  statement  as  to  how  the  soldier  can  use  a  protractor  to  measure  angles  and 
plot  azimuths,  but  the  ne.xt  screen  announces  that  this  lesson  will  not  actually  allow  him 
or  her  to  use  a  protractor.  [Why  not,  if  that  is  the  purpose  of  the  lesson?]  The  main 
point  here  is  that  the  long  lesson,  in  contrast,  doe,^  require  the  soldier  to  actually  use  a 
protractor.  Thus,  the  short  lesson  teaches  about  the  protractor;  the  long  lesson  teaches 
how  to  use  the  protractor.  The  skills  of  knowing  about  and  knowing  how  to  are  essen¬ 
tially  quite  different. 

Lesson  5c:  "Multiscalc  Gages."  In  this  case,  a  content  mismatch  between  long  and  short 
lessons  leads  to  problems  when  students  take  the  diagnostic  test.  (See  also  problem  area 
18.)  The  diagnostic  lesson  does  not  present  any  material  at  all  on  selecting  the  ap¬ 
propriate  range  or  setting  the  selector  switch  to  do  so.  Selecting  range  is  however 
covered  in  the  long  lesson.  In  fact,  the  "two  steps"  of  the  short  lesson  become  "three 
steps"  in  the  long  lesson: 
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Steps  to  reading  multimeters: 


(Compare:)  (short) 


(long) 


•  Select  the  correct  band. 

•  Determine  the  value  of  the  tick  mark  at  which  the  needle 
points. 

•  Select  the  correct  scale. 

•  Select  the  correct  range. 

•  Determine  the  value  of  the  tick  mark  at  which  the  needle 


points. 


Once  students  are  given  the  long  lesson  information  on  how  to  select  the  range,  they  are 
able  to  answer  all  test  items;  however,  with  the  short  lesson  material  only,  test  questions 
on  range  pose  major  difficulties. 


Lesson  36c:  "Capitalization."  The  lesson  presents  eight  simple  rules  for  capitalization. 
These  work  very  well  given  all  test  questions  except  items  seven  and  eight.  These  hvo 
items  require  application  of  lesson  rule  2.  As  stated,  this  rule  reads: 

Capitalize  the  first  word  of  a  direct  quote. 

This  is  an  ambiguous  and  incomplete  formulation,  since  it  does  not  cover  the  situations 
actually  presented  to  the  student  in  test  items  seven  and  eight.  This  rule  only  specifies 
that  the  first  word  of  a  direct  quote  need  be  capitalized;  however,  item  seven  contains  a 
quote  that  has  more  than  one  sentence.  In  item  eight,  the  quoted  sentence  is  divided 
into  two  separate  parts  with  the  normal  subjectA'erb/quotation  order  changed  to 
quotation/subjectA'erb/quotation  order  (i.e.  "that’s  how  I  did  it,"  he  said,  "by  climbing  a 
tree.")  Rule  2  does  not  tell  the  soldier  what  to  do  when  the  quoted  sentence  is  divided 
into  two  parts.  Should  only  the  first  word  of  the  first  half  be  capitalized?  What  about 
the  first  word  of  the  second  half?  Soldiers  mav  not  even  realize  that  we  are  dealing 
with  a  complete  though  divided  sentence  here.  Practice  has  not  illustrated  what  to  do  in 
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a  situation.  Moreover,  the  rule  does  not  specify  what  to  do  when  the  quote  precedes  the 
subject  phrase,  as  it  does  here. 

Lesson  27c:  "Scanning  for  Facts."  In  this  lesson,  rules  and  practice  items  are  insufficient 
for  answering  some  test  items,  although  the  mismatch  is  more  subtle  than  in  the  previous 
example.  In  the  lesson,  rules  and  numerous  practice  items  ask  the  student  to  identify 
mostly  one  or  two  word  clue  words  that  correspond  exactly  with  the  piece  of  information 
the  student  is  required  to  locate.  However,  on  "difficult"  test  items  the  required  answer 
is  usually  a  phrase  rather  than  one  or  two  words;  moreover,  in  these  items  the  "clue 
words"  are  more  complex  and  are  not  found  in  the  same  sentence  as  the  information  the 
student  is  required  to  locate.  Examples  follow. 

Item  1.  Asks  students  to  "touch  the  word  that  describes  what  perpendicular 
obstacles  do  to  the  movement  of  attackers." 

•  The  answer  is  not  "one  word"  --  as  slated  in  the  question  --  but  a  phrase. 

•  The  "clue  word"  here,  as  the  notion  has  been  used  in  the  lesson,  would  be 
"perpendicular  obstacles."  However,  in  this  case  "perpendicular  obstacles"  is 
insufficient;  it  must  be  combined  with  the  notion  of  "attackers"  as  well  as  a  notion 
of  effect  or  outcome  in  order  to  located  the  desired  information.  This  "clue"  is 
much  more  complex  than  those  given  in  the  lesson. 

•  The  required  information  is  not  found  in  the  sentence  containing  the  words 
"perpendicular  obstacles"  but  in  the  following  sentence.  This  is  a  condition  not 
taught  in  the  lesson,  where  clue  words  almost  always  act  as  flags  for  information 
located  in  the  same  sentence. 

In  item  8,  the  student  uses  the  clue  "dead  mail"  to  scan,  but  the  sentence  containing 
"dead  mail"  does  not  answer  the  question,  nor  does  the  following  sentence.  The  student 
must  continue  to  read  until  the  next  sentence.  Practice  in  this  sort  of  complex  scanning 
was,  in  fact,  not  given  in  the  lesson. 

Lesson  12h:  "Estimating  a  Sum  or  Difference."  The  short  lesson  does  not  review  material 
essential  for  answering  some  test  questions  concerned  with  the  place  value  to  which  a  sum 


or  difference  should  be  estimated.  Instruction  in  the  long  lesson  is  not  much  better; 
section  E,  "Choosing  a  Level  of  Estimation"  (which  appears  to  be  students’  main  weakness) 
gives  these  guidelines: 

Usually  if  you  are  working  with  smaller  numbers,  you  make  the  estimate  closer 
to  the  actual  numbers;  e.g.,  33  plus  36  is  about  70.  If  you’re  working  with 
larger  numbers,  the  estimate  would  not  be  as  close. 

If  you’re  in  a  hurry,  you’d  use  the  1st  or  2nd  two  digits  to  estimate. 

But  these  rules  do  not  explain  what  to  do  when  the  numbers  one  is  adding  or  subtracting 
have  different  place  values,  as  on  the  test.  Test  item  22,  for  example,  reads  as  follows: 

To  what  place  would  you  make  a  rough  estimate  of  these  numbers? 

17342  -  1618 

[This  is  poorly  worded:  the  question  intends  to  ask  for  an  estimate  of  the  difference  of 
these  numbers,  not  the  numbers  themselves.] 

The  answer  given  is  1,000. 

The  lesson  does  not  specify  how  to  determine  this.  It  has  only  taught  students  to  first 
round  the  two  numbers  separately  and  then  subtract  in  order  to  get  an  estimated  answer. 
Moreover,  the  lesson  contains  a  similar  practice  item  whose  answer  seems  to  contradict 
the  answer  given  for  this  test  item: 

(page  24,  short  lesson)  To  what  place  value  would  you  estimate  the  following? 

3252  +  79850 
Answer:  10,000. 

In  both  these  cases,  one  number  is  in  the  thousands  (and  presumably  is  rounded  to  the 
thousands)  and  the  other  is  in  the  ten  thousands  (and  is  presumably  rounded  to  the  ten 
thousands.)  In  both  cases  the  estimated  sum  or  difference  will  be  in  the  ten  thousands. 
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How  therefore  can  the  answer  be  1,000  in  the  first  case  and  10,000  in  the  second?  This 
is  inconsistent  -  either  the  test  answer  is  wrong  or  the  lesson  examples  are  wrong. 


The  source 


of  such  difficulties  may  be  in  the  language  of  the  question  itself,  in  the  instructions  for 


answering  the  question,  or  in  the  accompanying  diagrams  or  illustrations. 


Lesson  6b:  "Alining  Parts  and  EquipmcnL"  The  test  contains  some  very  ambiguously 
drawn  items,  as  well  as  awkward  and  ambiguous  language.  The  lesson  introduction  itself 
warns  of  problems  to  come: 


In  this  lesson  pictures  of  objects  will  appear  on  the  screen.  Your  task  will  be 
to  aline  objects  as  directed.  You  will  use  arrows  to  move  the  objects,  instead 
of  actually  moving  them  around.  [As  if  objects  pictured  on  a  screen  could 
actually  be  moved  around....] 


In  test  items  9,  11,  and  12  confusing  diagrams  and  instructions  make  it  very  difficult 
to  answer  correctly.  In  item  11,  for  example,  there  are  two  ways  to  make  a  45  degree 
angle.  In  item  12,  the  language  as  well  as  the  diagram  is  confusing.  This  item  requires 
the  student  to  "move  the  right  edge  of  the  logs  down  . . . ."  (Do  logs  have  "left"  or 
"right"  edges?  What  are  the  logs  in  the  illustration?)  This  item  also  seems  to  require 
previous  knowledge  of  terminology  such  as  "culverts"  and  "clogs." 


Lesson  27c:  "Scanning  for  Facts."  Aside  from  aforementioned  problems  with  lesson/test 
content  mismatch,  some  test  items  are  simply  misleading.  Item  9,  for  example,  asks  the 
student  to  scan  for  the  answer  to  the  question,  "When  should  a  native  scout  lead  a 
patrol?"  Based  on  the  way  the  question  is  phrased,  one  expects  to  find  some  mention  of 
conditions  or  time  in  which  native  scouts  should  lead.  However,  the  answer  to  the 
question  is  "never."  During  rapid  scanning  it  is  very  difficult  to  identify  "never"  as  the 
required  "when."  I  had  trouble  answering  this  one. 


Lesson  26a:  "Recognizing  Word  Meanings.”  One  test  item  has  two  possible  answers. 


^For  purposes  of  test  security,  the  items  shown  here  resemble,  but  are  not  the  same 
as,  actual  JSEP  items. 
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Lesson  21c.  "Finding  Information  in  a  Book."  Test  item  31  is  ambiguous: 

SGT  Smith  needs  to  locate  the  page  numbers  of  a  specific  subject  in  his  TM  manual. 
Which  part  of  the  book  should  he  use? 

•  The  appendix 

•  The  glossary 

•  the  index 

•  the  table  of  contents 

This  question  does  not  define  "specific  subject"  adequately  enough  to  allow  a  choice  to  be 
made  among  these  options. 

Item  32  appears  to  have  a  wrong  answer.  The  question  asks  the  student  to  select  the 
term  that  describes  the  section  of  a  book  that  contains  any  related  material  that  is  not 
part  of  the  text.  The  answer  accepted  by  the  program  is  "glossary."  However,  a  glossary 
is  better  described  as  a  set  of  definitions  for  specialized  terms;  the  correct  answer  should 
be  "appendix." 

Lesson  15d:  "Use  a  Protractor."  Test  items  in  this  lesson  have  typographical  errors  and 
omissions  that  lead  to  confusing  language.  Item  31  asks  students  to  identify  the  steps  in 
in  ". . .  mcasring  [sic]  an  angle  with  a  protractor." 

Two  answer  choices  read: 

•  Line  up  the  reference  line  of  the  protractor  with  1  of  the  angles,  ["line" 
apparently  omitted  here  before  "1";  only  one  angle  is  pictured.] 

•  Read  the  protractor  scale  where  the  vertex  crosses  it.  [awkward  phrasing;  a 
"vertex"  does  not  "cross"  the  scale.] 

Item  32  is  basically  the  same  but  asks  for  the  second  step  in  "measring"  [sic]  an  angle. 
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Chapter  3.  The  JSEP  Curriculum— General  Considerations 


"Successful"  vs.  "Unsuccessful"  Lessons;  What  Makes  a  Lesson  "Good"? 

Since  the  purpose  of  the  preceding  review  was  to  identify  problem  areas  in  JSEP 
lessons  in  such  a  way  as  to  provide  a  basis  for  revision  and  correction,  the  comments 
made  reflect  a  deliberate  hunt  for  flaws  rather  than  a  balanced  judgment  of  JSEP  lessons. 
Though  many  of  the  lessons  reviewed  had  at  least  one  problem,  some  lessons  did  not  have 
any  and  could  therefore  be  considered  "good"  lessons.  In  particular,  of  those  lessons 
reviewed,  7a  --  Identify  Points,  Lines,  Segments,  and  Rays,  and  30a  --  Identify  Flow 
Chart  Symbols,  could  be  considered  very  good  examples  of  instructional  design. 

The  primary  characteristic  of  these  lessons  as  opposed  to  others  was  that  (1)  their 
lesson  design  was  not  step-  or  rule-based  and  (2)  the  lesson  design  put  the  capabilities  of 
the  computer  as  delivery  system  to  best  use.  The  objective  of  both  lessons  was  to  teach 
students  to  identify  or  recognize  shapes.  In  lesson  7a.  the  major  terms  point,  line,  ray, 
etc.,  were  illustrated  and  accompanied  by  an  on-screen  verbal  definition.  Similarly,  lesson 
30a  illustrated  each  flow  chart  symbol  along  with  a  statement  of  its  significance.  In  both 
lessons,  the  objective  was  to  teach  a  one-to-one  correspondence  between  an  object  and  its 
meaning.  An  on-screen  format  was  ideal  for  this  purpose,  because  of  the  simple  cor¬ 
respondence  between  the  graphic  image  and  its  verbal  identifier.  Practice  questions  in 
lesson  30a,  for  example,  used  a  basic  stimulus  --  recall  —  reinforcement  model,  in  which 
students  were  asked  to  recall  the  meanings  of  forms  pictured  on-screen.  After  every 
three  symbols,  a  review  section  tested  students’  recall  of  all  of  the  preceding  symbols, 
thus  reinforcing  previous  learning. 

The  objectives  of  these  two  lessons  were  limited  to  low-levcl  identification  skills  and 
did  not  require  more  complicated  mathematical  or  verbal  processing.  Thus  they  could 
avoid  the  pitfalls  of  the  step  approach,  inappropriate  surface  featurcs-focus,  and  lack  of 
concept  e.vplanations  common  in  other  lessons.  These  "good"  lessons  taught  what  they 
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intended  to  teach,  since  the  skill  to  be  taught  (identification)  was  already  at  the  most 
basic  level.  There  was  thus  no  occasion  for  skill  confusion  such  as  that  seen  in  other 
JSEP  lessons  where  the  "intended"  higher-level  skill  was  broken  down  and  taught  only 
peripherally  in  terms  of  "basic"  skill  components.** 

JSEP  Assumptions  About  Learners:  User  Choice  and  Program  Flexibility 

A  major  issue  that  arose  in  the  course  of  the  curriculum  evaluation  concerns  the 
status  of  user  choice  and  responsibility  for  learning.  The  JSEP  curriculum  as  it  currently 
stands  offers  little  opportunity  for  students  to  make  real  choices  in  their  interaction  with 
the  program.  All  major  responsibilities  for  lesson  selection  and  progress  through  lessons 
rest  with  the  soldier  management  system  rather  than  with  the  individual  soldier.  Even 
though  students  can  decide  the  order  of  the  lessons  they  take  or  the  order  of  the  lesson 
sub-sections  in  long  lessons,  there  are  few  opportunities  for  students  to  exercise  truly 
responsible  choice  -  that  is,  to  make  decisions  about  whether  they  need  to  take  a 
particular  lesson  or  section  of  a  lesson,  or  which  parts  they  need  to  take  over  again,  or 
which  lessons  they  want  to  take  despite  having  passed  the  pretest.^  The  only  occasion  on 
which  there  is  some  freedom  to  make  such  decisions  is  in  long  lessons  that  offer  the 
student  the  choice  between  more  tutorial  instruction  or  more  practice.  There  are 
numerous  points  in  the  program  at  which  soldiers  could  be  given  more  responsibility  for 
their  own  learning.  Overall,  the  JSEP  curriculum  design  makes  an  assumption  about 
learners  that  seems  to  reflect  a  limited  view  of  students’  abilities  to  make  judgments 
about  their  own  academic  progress  or  performance. 

Student  comments  on  this  topic  reveal  that  they  are  both  able  and  willing  to 
exercise  such  choices.  In  the  attitude  survey,  students  said: 


‘‘This  does  not  mean  that  a  computer-based  instructional  system  such  as  JSEP  is  only 
good  for  teaching  the  most  "basic"  or  low-level  skills.  Indeed,  higher-order  skills  can  be 
taught,  but  this  requires  preserving  the  essential  concept  behind  the  skill,  not  replacing  it 
with  lower-order  peripheral  skills.  In  the  JSEP  lessons  I  reviewed,  however,  higher-level 
skills  are  too  often  reduced  or  broken  down  into  steps  involving  lower-level  skills  which, 
taken  as  a  whole,  do  not  lead  to  an  adequate  conceptual  understanding  of  underlying 
notions  or  processes  behind  skills. 

^'The  lesson  pretests  were  instituted  as  part  of  the  JSEP  evaluation  plan,  in  order  to 
provide  a  baseline  against  which  to  measure  students’  subsequent  learning  of  lesson  material. 
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"After  you  take  the  first  posttest  before  a  lesson,  you  should  be  able  to  say  ‘yes’  or 
‘no’  to  taking  the  test  over  before  getting  into  the  lesson.  Sometimes  you  missed  it 
by  one  and  you  know  the  one  you  missed." 

"I  would  make  it  so  that  if  an  individual  wanted  to  get  more  practice  he  could  go 
back  to  a  specific  lesson  and  work  on  weaknesses." 

In  personal  notes,  students  said; 

"If  we  are  about  to  choose  a  lesson  we  should  have  more  control  of  the  system  than 
we  do.  For  example,  I  have  just  finished  a  lesson  and  I  have  only  20  minutes  left. 

I  do  not  wish  to  start  a  paper-based  lesson,  it  being  Friday,  because  I  will  not  have 
enough  time  to  finish  the  lesson  today,  and  I  will  have  to  jump  right  into  it  on 
Monday  and  I  would  rather  have  a  lesson  that  I  could  completely  finish  today.  But 
the  way  the  system  is  set  up  that  is  not  possible  for  me  to  do  that  I  am  aware  of, 
once  I  select  a  lesson  that  is  it  I  have  to  finish  it  to  get  out  of  it.  That  does  not 
give  us  much  room  for  time  management.  Please  if  there  is  a  way  to  allow  this  it 
would  really  make  it  easier  for  me  to  manage  my  lessons  better." 

"I  could  solve  the  problems  but  don't  understand  the  process.  The  only  way  that  I 
could  take  the  lesson  is  by  failing  the  tests." 

[This  student  obviously  would  like  a  choice  to  go  through  the  lesson  or  not,  despite 
having  passed  the  test.] 

"I  did  the  best  I  could  on  this  task  and  I  passed.  I  had  to  guess  some  of  the 
questions  and  got  lucky.  I  need  to  get  into  the  task." 

[Similarly,  this  student  recognizes  a  need  for  more  instruction  and  would  like  to  be  able 
to  take  the  lesson.] 

"I  should  be  able  to  get  into  the  lesson  if  I  feel  unsure."  [Same  thing  here.] 

"Let  it  be  my  choice  to  go  into  a  lesson  or  not." 
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"The  student  should  be  allowed  to  go  back  over  the  lesson  even  if  he  passed  it." 

It  is  obvious  from  these  comments  that  students  realize  when  they  are  unsure  of 
lesson  material,  and  when  they  would  benefit  from  more  instruction  or  from  a  chance  to 
review.  As  it  stands  now,  the  JSEP  program  is  not  flexible  enough  to  address  these 
desires,  and  ignores  students’  ability  to  assess  their  own  instructional  needs. 

Allowing  students  more  flexibility  in  their  progress  through  the  program  would  also 
enhance  the  "diagnostic"  character  of  JSEP.  Perhaps  most  importantly,  the  more  an 
instructional  program  encourages  students  to  take  responsibility  for  their  own  learning  by 
being  flexible  enough  to  meet  their  needs  and  desires,  the  better  its  subjective  psycholog¬ 
ical  impact  and,  in  the  long  run,  its  instructional  effectiveness. 


JSEP  Assumptions  About  Learners:  The  Test  Feedback  Issue 

Another  area  in  which  JSEP  lesson  design  is  based  on  a  crucial  unexamined  assump¬ 
tion  is  in  the  area  of  test  feedback.  According  to  designers,  students  should  not  be 
allowed  to  know  the  answers  to  tests  because  this  would  compromise  test  security. 
Repeated  observations,  interviews,  and  student  comments  in  surveys  and  personal  notes 
indicate  that  this  is  an  unfounded  fear.  Students  who  take  JSEP  are  nei  interested  in 
beating  the  system;  they  are  simply  interested  in  learning  as  much  as  possible  from  the 
program.  To  assume  that  telling  students  which  questions  they  missed  will  lead  them  to 
pass  along  answers  to  their  colleagues  reflects  a  lack  of  faith  in  students’  desire  to 
improve  themselves,  as  well  as  a  somewhat  "grade-schoolish"  attitude  toward  learning  that 
is  simply  incompatible  with  a  computer-based  instructional  system  in  the  first  place. 

For  a  test-based  program  such  as  JSEP,  it  is  unfair  as  well  as  unwise  to  forbid 
students  the  chance  to  see  where  they  made  their  mistakes.  Students’  major  request  is: 
"Please  let  us  see  which  questions  we  missed  so  we  can  improve  ourselves."  In  nearly 
every  other  test  situation  in  which  the  purpose  is  educational  improvement  (and  not 
merely  assessing  performance  or  rank  relative  to  others)  they  receive  such  feedback. 

There  is  no  reason  why  JSEP  should  not  allow  students  a  similar  opportunity.  It  simply  is 
not  true  that  a  student  can  tell  which  test  questions  were  missed  by  looking  at  perfor¬ 
mance  on  practice  questions.  Students  have  commented:  "I  can  pass  every  practice 
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question  perfectly  and  still  flunk  the  test."  Given  the  already  described  problems  with 
test  items,  this  is  not  surprising.  As  long  as  JSEP  remains  a  test-oriented  instructional 
program  -  especially  one  in  which  there  are  various  ways  students  can  make  mistakes  on 
test  items,  aside  from  simply  not  knowing  the  skill  --  it  is  unreasonable  not  to  offer 
students  a  chance  to  see  their  errors. 


JSEP  and  Underlying  Targeted  Competencies 

The  JSEP  curriculum  is  supposed  to  provide  remedial  instruction  in  targeted  com¬ 
petencies  for  selected  MOSs.  Though  there  are  186  different  JSEP  lessons  dealing  with 
these  specific  competencies,  there  appears  to  be  a  small  number  of  unstated  competencies 
that  are  actually  taught  by  the  lessons.  This  review  has  identified  at  least  three  basic 
competencies  (aside  from  reading)  that  underlie  the  skills  supposedly  taught  by  JSEP 
lessons.  These  are  identification  (or  recognition),  element  manipulation,  and  pattern 
recognition.  Often  enough,  as  described  earlier  in  problem  areas  1  and  2,  the  lessons 
teach  these  skills  rather  than  the  stated  objective  of  the  lesson.  For  example,  even  in 
lessons  whose  focus  is  on  teaching  a  skill  such  as  rounding,  a  large  proportion  of  the 
lesson  is  spent  on  identifying  and  naming  problem  elements  such  as  place  values.  The 
focus  on  clement  manipulation  is  seen  in  lessons  that  require  the  student  to  solve 
problems  entirely  by  the  application  of  given  rules  of  mechanical  relation  between  problem 
elements.  Students  are  not  required  to  "understand"  the  task,  but  merely  to  apply  the 
rule  correctly  in  order  to  generate  a  correct  answer.  Many  of  the  quantitative  lessons  in 
the  JSEP  scries  involve  such  element  manipulations. 

Tasks  requiring  the  skill  of  pattern  recognition  ask  the  student  to  match  up  numbers, 
compare  series  of  numbers,  compare  printed  information,  or  copy  information  verbatim. 
This  skill  is  taught  in  JSEP  lessons  such  as  outlining,  which  depends  more  on  recognition 
and  use  of  a  patterned  code  than  upon  higher  order  skills  such  as  deciding  the  relative 
level  of  abstraction  of  textual  elements. 

There  undoubtedly  are  other  examples  of  underlying  competencies  taught  in  the 
lessons,  but  the  purpose  here  has  been  to  give  e.xamples  of  cases  where  lower-order  skills 
substitute  for  a  focus  on  higher  order  skills  that  are  in  fact  necessary  to  perform  the 
targeted  skill  of  a  lesson.  The  fundamental  instructional  problem  with  such  lessons  is 
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that,  while  aiming  for  simplicity  and  ease  of  instruction,  the  lessons  have  also  avoided 
teaching  the  notions  that  are  most  important  (and  ultimately  most  helpful)  to  students  in 
performance  of  the  targeted  skill.  In  general,  the  skills  that  get  overlooked  in  JSEP 
lessons  arc  the  higher  order  skills  such  as  inference,  abstraction,  synthesis,  association,  or 
evaluation.  These  are  essential  to  performance  of  many  targeted  competencies,  and  yet 
these  are  the  skills  that  most  "lower  ability"  students  lack.  They  are  therefore  the  ones 
that  should  be  taught  in  the  lessons. 

This  is  shown,  for  instance,  in  the  lesson  on  finding  information  in  a  book.  The 
lesson  teaches,  and  gives  lots  of  helpful  practice  in,  locating  page  numbers  for  specific 
points  of  information.  But  this  is  not  the  main  skill  in  fact  needed  in  order  to  find 
information  in  a  book.  Rather,  a  student  needs  to  learn  how  to  find  a  given  piece  of 
information  based  on  the  characteristics  of  that  information  --  topic,  specificity  --  and  to 
compare  these  with  what  he  or  she  knows  about  the  different  characteristics  of  informa¬ 
tion  presented  in  a  table  of  contents,  a  glossary,  index,  etc.  In  order  to  do  this, 
however,  higher-order  skills  involving  assessment  of  levels  of  abstraction  and  association 
are  needed.  There  is  no  instruction  in  fact  on  the  "differences"  in  the  information 
presented  in  each  of  these  text  sections.  Thus  the  student  does  not  learn  the  essential 
skills  based  on  a  conceptual  understanding  that  are  essential  to  performance  of  the 
targeted  skill,  locating  information  in  a  book.  When  this  occurs.  lessons  do  not  in  fact 
teach  what  they  intend  to  teach,  but  teach  only  supporting  or  peripheral  skills. 

Higher-order  skills  may  not  be  intrinsically  more  difficult  to  learn,  contrary  to  the 
assumption  made  by  most  basic-skills  oriented  curricula.  They  can  be  taught  in  a 
simplified  manner;  but  it  is  essential  to  avoid  confusing  "simplified  manner"  with  simplific¬ 
ation  of  the  skill  itself,  or  reduction  into  component  skills  that  when  added  together 
mimic  performance  of  the  higher  order  skill.  This  seems  to  have  been  a  major  problem  in 
some  JSEP  lessons,  where  attempted  use  of  simplified  language  or  approach  seems  to  have 
led  to  an  avoidance  of  concept  explanations,  perhaps  for  fear  that  these  would  complicate 
the  lesson.  Yet,  avoidance  of  concept  explanations  tends  to  increase  the  multiplication  of 
rules  and  steps  based  on  simpler  tasks  such  as  pattern  recognition  or  identification, 
ultimately  leading  not  to  simplicity,  but  to  more  opportunity  for  confusion  and  errors. 

In  order  to  do  an  adequate  Job  of  teaching  skills  that  can  be  applied  to  jobs  or  that  can 
contribute  to  job-related  learning,  it  is  essential  that  JSEP  lessons  provide  a  conceptual 
basis  for  what  is  being  taught. 
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JSEP  and  Field  Dependent  Thinking 


A  related  problem  concerns  what  educators  term  "field  dependency."  This  notion 
describes  the  tendency  of  some  (mostly  lower-ability)  students  to  remain  "grounded"  in 
their  thinking.  Students  who  are  field  dependent  have  difficulty  associating  what  they 
have  learned  in  one  context  with  another  related  context.  They  are  unable  to  make  the 
necessary  cognitive  jumps  from  information  or  situations  as  given,  to  the  application  of 
the  same  information  to  new  situations.  Most  JSEP  lessons  reviewed  encourage  students 
to  remain  at  this  basic  field  dependent  level  in  their  approach  to  tasks.  The  lessons 
repeatedly  remind  students  to  perform  all  the  steps  in  a  task  exactly  as  given,  without 
encouraging  combinations  or  abbreviations  of  steps  or  other  ways  to  save  time.  The 
practice  questions  do  not  require  students  to  apply  their  learning  to  new  situations  or 
contexts;  rather,  they  present  the  same  problem  formats  over  and  over  again.  By  failing 
to  offer  students  practice  in  moving  away  from  very  grounded,  situation-  and  step-based 
thinking  in  approaching  tasks,  JSEP  primarily  offers  practice  in  thinking  in  simplest  terms. 
It  is  questionable  whether  this  should  in  fact  be  the  endpoint  of  any  instruetional 
program.  Lower-ability  students  do  not  need  more  encouragement  to  remain  grounded  in 
their  thinking;  they  need,  on  the  contrary,  encouragement  to  get  beyond  this  basic  level. 
The  real  strength  of  any  instructional  program  is  the  extent  to  which  it  can  develop  the 
learners  ability  to  go  beyond  the  information  presented  to  more  sophisticated  levels  of 
concept  application  and  analysis.  As  it  stands,  JSEP  encourages  the  soldier  to  consider 
only  the  information  given  in  the  most  straightforward,  literal  way.  It  does  not  suggest 
that  there  may  be  alternative  interpretations  or  approaches  to  problems  or  evidence. 

A  simple  example  of  this  is  found  in  the  Time  Management  lesson.  The  lesson 
approaches  the  concept  of  "time  on  task"  in  a  very  literal  manner,  obscuring  the  neces¬ 
sary  interpretive  skills  involved  in  making  a  judgment  about  time  on  task.  Two  soldiers 
are  shown  talking  while  sitting  at  (presumably  JSEP)  computer  terminals.  The  student 
must  touch  the  box  indicating  whether  this  illustration  represents  "on-task"  or  "off-task" 
behavior.  The  required  answer  is  "off-task."  However,  this  answer  does  not  allow  the 
alternative  interpretation  that  the  soldiers  were  actually  talking  about  JSEP  lessons,  rather 
than  about  non-related  subjects  --  in  which  case  their  talk  would  have  been  entirely  "on- 
task."  The  concept  of  "on-task"  has  been  reduced  to  its  most  literal,  appearance-based 
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level,  ignoring  the  fact  that  alternative  judgments  could  be  made.  The  supplementary 
feedback,  "You  will  not  need  to  talk  to  someone  to  complete  JSEP  lessons"  is  also 
simplistic;  students  may  very  well  need  to  talk  to  someone  --  indeed,  they  often  do  -- 
while  they  complete  JSEP  lessons.  Moreover,  the  implicit  message,  "Don’t  talk  to  others 
while  you  are  on  JSEP"  defeats  what  the  computer-assisted  instruction  literature  considers 
one  of  the  major  advantages  of  computers  as  an  instructional  medium  -  the  way  they 
encourage  peer  interaction  in  solving  problems.  Computers  encourage  users  to  help  each 
other  and  to  learn  from  each  other  --  yet  this  advantage  is  explicitly  denied  in  the  way 
soldiers  have  been  encouraged  to  think  about  "time  on  task." 


Subjective  Appeal  and  Paper-Based  Lessons 

One  of  the  major  positive  points  about  the  JSEP  curriculum  is  in  the  area  of  on¬ 
screen  delivery.  Aside  from  touch-screen  problems,  graphics  that  take  too  long  to 
complete  (soldiers  made  repeated  complaints  about  these  two  aspects),  and  occasionally 
confusing  or  distracting  illustrations,  on-screen  presentation  of  lesson  material  was 
appealing.  Use  of  screen  space  was  effective,  with  screens  neither  overloaded  nor  too 
empty.  Interactiveness  was  generally  maintained,  with  students  frequently  required  to 
respond  in  one  way  or  another  to  screen  material,  without  long  stretches  of  "page 
turning."  The  subjective  appeal  of  interacting  with  a  computer  was  undoubtedly  a  major 
factor  in  students’  preferences  for  JSEP  over  BSEP  and  in  students’  overall  sense  of 
satisfaction  with  the  program. 

However,  according  to  student  comments,  the  appeal  of  the  paper-based  JSEP  lessons 
was  significantly  less  than  that  of  the  on-line  delivery  system.  These  paper  lessons  were 
designed  to  cover  prerequisite  competencies  involving  complicated  graphics  or  tasks  (such 
as  writing)  that  would  have  been  unsuitable  for  on-line  presentation.  However,  switching 
from  on-line  to  paper  delivery  proved  to  be  cumbersome  for  both  teachers  and  students. 
Students  said  they  "wasted  time"  signing  on  and  off  the  computer;  teachers  disliked  the 
grading  and  record-keeping  involved.  Significant  errors  and  organizational  problems  were 
noted  in  the  lessons  reviewed;  in  addition,  some  of  the  lessons  (for  example,  the  learning 
strategies  paper-based  lesson  for  Problem  Solving)  were  far  too  lengthy.  Also,  for  some 
lessons  it  was  not  immediately  apparent  why  the  contents  of  the  lesson  were  not  suitable 
for  on-line  format.  (For  example.  Lesson  11a,  "Identify  shape  and  position  terms,"  and 
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the  long  lesson  for  28b,  "Finding  information"  contained  material  that  appeared  to  be  quite 
similar  to  that  of  other  on-line  lessons  involving  shape  identification  and  reading  charts 
and  tables.  With  some  changes  (especially  in  the  detail  of  the  tables  given  in  28b),  these 
lessons  would  appear  suitable  for  on-line  delivery. 

JSEP  can  undoubtedly  make  a  contribution  both  to  Army  education  and  to  the 
development  of  individual  soldier  competencies.  It  can,  however,  make  such  a  contribution 
much  more  efficiently  and  effectively  if  certain  of  the  problems  identified  in  this  review 
are  corrected.  To  some  extent  the  problems  identified  are  not  specific  to  JSEP  or  to 
computer-based  instruction  alone;  their  source  lies  in  the  underlying  assumptions  and 
approaches  currently  in  vogue  in  models  of  basic  skills  or  remedial  instruction.  As 
educators  and  instructional  designers  are  aware,  instructional  materials  and  programs 
themselves  are  not  paradigm-free,  and  sometimes  they  represent  more  the  opinions  of  the 
"experts"  about  what  learning  should  be  like,  than  the  realities  of  what  learning  in  fact  is 
like  for  many  students. 

Yet  the  responsibility  of  instructional  designers  to  bring  their  materials  ever  closer 
to  meeting  students’  needs  cannot  be  dismissed.  Perhaps  by  taking  steps  toward  col¬ 
laboration,  by  involving  students  more  deeply  in  the  design  and  revision  process,  and  by 
in-depth  examination  of  "hidden"  as  well  as  obvious  instructional  strategies  and  needs, 
commonly  defined  goals  of  excellence  can  be  set  and  achieved. 


Chapter  4.  JSEP  Supporting  Materials  Review 


This  chapter  offers  summary  comments  on  four  important  supporting  materials  for  the 
JSEP  program:  the  JSEP  Learning  Strategics  Lessons,  the  JSEP  Instructor’s  Manual,  the 
Soldier  Management  System  ISMSV-Features  and  Functions  Guide,  and  the  JSEP  Test.® 
These  materials  were  designed  to  aid  JSEP  instructors  in  the  performance  of  tasks 
required  by  their  role  as  instructors  and  program  managers,  and  to  help  students  in 
learning  JSEP  lesson  content.  The  primary  focus  of  the  review  comments  will  therefore 
be  upon  how  well  these  materials  meet  criteria  for  helpfulness.  Specifically,  this  review 
considers  the  following  questions: 

For  Manuals: 

•  Do  the  materials  have  a  clearly  stated  purpose,  and  do  the  contents  reflect  this 
purpose? 

•  Are  the  materials  organized  so  that  users  can  easily  follow  points  made  and 
locate  desired  information?  Is  all  essential  information  included? 

•  Is  the  language  free  of  error,  clear,  and  easy  to  understand? 

•  Is  the  information  presented  in  a  format  that  encourages  users,  answers  their 
questions,  and  inspires  further  involvement  with  the  JSEP  program? 


®This  section  summarizes  more  detailed  reviews.  The  reviews  of  draft  versions  of 
both  manuals  and  the  JSEP  test  were  submitted  to  the  Army  Research  Institute. 
Presumably  these  materials  will  be  revised  before  their  final  release. 
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For  the  Learning  Strategies  Lessons: 


•  Are  the  strategies  taught  in  such  a  way  as  to  encourage  transfer  of  learning  to 
JSEP  lessons? 

•  Is  the  lesson  purpose  clear  and  well-illustrated? 

•  Does  the  lesson  in  fact  teach  students  material  that  facilitates  learning  how  to 
learn? 

For  the  JSEP  Test: 

•  Are  the  items  clearly  written,  free  of  errors  or  ambiguities? 

•  Do  the  items  in  fact  test  what  they  intend  to  test? 

•  Are  accompanying  graphics  easy  to  read? 


JSEP  Learning  Strategies 


Purpose 

This  section  reviews  one  of  the  five  JSEP  Learning  Strategies  lessons,  the  Problem 
Solving  module.  Since  these  lessons  were  designed  to  stand  independently  of  the  JSEP 
subject  matter  curriculum  and  since  they  were  reviewed  internally  by  Army  Research 
Institute  personnel,  a  systematic  review  of  each  was  not  performed.  However,  because 
many  students  actually  took  at  least  a  portion  of  the  Problem  Solving  module,  review 
comments  are  included  here  in  order  to  provide  a  more  complete  look  at  how  the  JSEP 
curriculum  as  a  whole  functions. 

Rationale  for  Learning  Strategies  Lessons 

The  intention  of  the  learning  strategy  lessons  is  to  provide  remedial  help  to  soldiers 
who  experience  difficulty  with  the  JSEP  program  by  teaching  them  how  to  go  about 
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approaching  the  learning  tasks  that  are  found  in  JSEP  lessons.  The  basic  assumption  is 
that  students  can  learn  better  if  they  first  learn  how  to  leam  --  that  is,  if  they  have  a 
set  of  strategies  that  they  can  apply  to  different  problem  situations  to  facilitate  their 
interaction  with  the  material.  This  principle  has  been  translated  in  the  JSEP  learning 
strategies  model  into  a  set  of  lessons  detached  from  the  main  curriculum.  Soldiers  are 
routed  into  these  lessons  if:  (1)  they  have  been  prescribed  by  the  JSEP  instructor,  or  (2) 
the  soldier  management  system  detects  repeated  failures  or  other  problems  in  student 
performance  that  indicate  that  one  of  the  strategies  is  needed. 

This  model  of  "detached"  instruction  seems  reasonable.  However,  as  R.  P.  Kern  notes 
in  his  analysis  of  the  Reading  Strategies  module^,  the  detached  learning  premise  is  flawed 
when  applied  to  the  Army  situation,  for  there  is  little  or  no  evidence  for  the  trans¬ 
ferability  of  learning  from  generalized  non-Army  relevant  content  to  Army-relevant 
content.  Kern  also  notes  that  printed  self-study  materials  (such  as  those  found  in  the 
Reading  Strategies  lesson  and  others)  do  not  seem  to  lead  to  much  effective  learning 
among  lower-ability  students.  He  recommends  that  the  Reading  Strategies  lesson  be 
reformatted  as  "help"  sheets  whose  practice  content  is  taken  directly  from  JSEP  lesson 
material.  These  help  sheets  would  appear  as  aids  to  students,  available  when  needed,  and 
not  as  a  "front  end  block  of  instruction."  [Kern,  p.  3] 

Problem  Solving  Lesson  -  Purpose  and  Format 

The  apparent  purpose  of  the  Problem  Solving  module,  according  to  the  on-screen 
statement  of  lesson  purpose,  is  to  teach  students  to  ". . .  learn  more  effective  ways  of 
problem  solving  and  show  how  to  identify  different  types  of  word  problems."  This  purpose 
is  somewhat  unclear,  since  the  first  part  of  the  statement  refers  to  problem  solving  in 
general,  and  the  second  part  refers  to  math  word  problems. 

According  to  information  provided  by  staff  at  the  Army  Research  Institute  and 
Florida  State  University,  the  module  has  five  lessons,  with  the  first  two  lessons  on-line, 
and  the  other  three  in  paper  format.  The  five  lessons,  as  listed  at  the  beginning  of  the 
paper-based  lesson  5,  are: 


"See  R.P.  Kern’s  review  of  the  Reading  Strategies  Module  for  JSEP,  May  1987. 
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Lesson  1:  Introduction  (on  line) 

Lesson  2:  Identifying  Problem  Types  (on  line) 

Lesson  3:  Goals  and  Diagrams  (paper) 

Lesson  4:  Selecting  Sub-Goals  (paper) 

Lesson  5:  Answer  Recognition  (paper) 

In  addition,  there  is  a  "Math  Prompt  Notebook"  to  be  used  when  the  on-line  prompt 
for  problem  solving  appears  while  a  student  is  working  on  a  math  lesson. 

According  to  my  review  of  the  module,  the  material  from  lesson  2  has  been  combined 
with  that  of  lesson  1  to  form  a  single  on-line  unnumbered  lesson.  This  change  has  not, 
as  far  as  I  have  been  able  to  tell,  been  noted  anywhere  in  the  lesson  materials  themsel¬ 
ves.  In  fact,  the  instructions  for  lesson  3  remind  the  student  that  he  or  she  must  first 
complete  "lesson  1"  and  "lesson  2."  The  on-line  lesson  does  not  describe  the  various  parts 
of  the  module,  so  the  soldier  has  no  idea  of  the  general  topics  to  be  covered.  Neither  is 
there  any  indication  that  there  are  paper  lessons  that  must  be  completed  after  the  on-line 
portion.  In  fact,  at  the  end  of  the  on-line  lesson  the  soldier  is  told:  "You  have  com¬ 
pleted  the  material  on  problem  solving.  Thank  you  . . . ."  It  would  appear  that  there  is 
a  serious  discontinuity  in  the  module  design. 

Education  staff  contacted  at  evaluation  sites  said  only  the  on-line  portion  of  the 
module  was  used.  Staff  members  at  two  sites  said  they  had  the  vaper  lessons,  but  did  not 
use  them.  (One  did  use  the  math  prompt  notebook,  however.)  At  another  site,  instruc¬ 
tors  said  they  did  not  have  any  paper  lessons  for  the  module. 

General  Module  Review  Comments 

This  module  was  difficult  to  evaluate,  because  it  was  not  clear  exactly  what  it 
consisted  of.  In  principle,  the  soldier  takes  all  five  lessons;  in  reality,  he  or  she  probably 
takes  only  the  on-line  lesson,  and  perhaps  uses  the  math  prompt  notebook.  If  the  soldier 
takes  only  the  on-line  portion  (which  deals  primarily  with  problem  type  identification)  the 
module  does  not  teach  what  it  intends  to  teach  -  for  the  soldier  cannot  learn  strategies 
for  solving  problems,  but  only  how  to  identify  types  of  problems.  These  skills  are 
definitely  not  the  same. 
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Second,  the  relationship  of  the  module  to  the  JSEP  curriculum  as  a  whole  is  proble¬ 
matic.  The  module  is  focused  almost  entirely  on  solving  math  word  problems,  yet  there 
are  few  lessons  on  math  word  problems  in  JSEP.  (In  fact,  JSEP’s  lack  of  instruction  in 
word  problems  was  cited  as  a  weakness  by  some  instructors  and  students.)  The  JSEP 
lessons  that  do  deal  with  word  problems  cover  only  simple  ones.  Yet,  all  of  lesson  4  in 
the  Problem  Solving  module  concerns  complex  word  problems. 

Third,  the  student  is  assigned  to  the  Problem  Solving  module  when  he  or  she  fails 
five  math  lessons,  whether  or  not  these  lessons  involved  solving  math  word  problems. 
Thus,  much,  if  not  all,  of  the  material  in  the  module  may  be  irrelevant  to  students’  needs. 

Another  consideration  relates  to  the  length  of  the  module.  Requiring  the  student  to 
go  through  157  pages  of  paper-based  material  (in  addition  to  the  on-line  module  material) 
is  simply  not  realistic.  Even  students  who  start  out  highly  motivated  are  likely  to  have  a 
hard  time  remaining  so  when  confronted  with  such  long  lessons. 


One  of  the  main  concerns  in  discussions  of  remedial  skills  education  concerns  the 
issue  of  context,  with  much  of  the  research  pointing  to  lack  of  transferability  of  skills 
from  one  context  to  another.  This  is  seen  (by  Kern  and  others)  as  an  argument  for 
"embedded"  instruction  --  instruction  based  on  the  actual  materials  the  student  is  working 
with  "  rather  than  "detached"  instruction,  such  as  that  represented  by  the  JSEP  learning 
strategies  lessons.  Following  this  argument,  the  Problem  Solving  module  should  use 
examples  drawn  from  the  lesson  material  students  are  actually  working  with,  rather  than 
generalized  content  that  is  not  related  to  JSEP.  Skill  learning  would  thereby  be  en¬ 
hanced,  since  the  student  can  more  readily  apply  it  to  the  job  conte,xts  in  which  the  skills 
must  be  used. 

There  is  a  difficulty  with  the  transferability  argument,  however,  that  most  discus¬ 
sions  of  the  problem  ignore.  Transferability  only  becomes  significant  when  one  considers 
first  what  students  are  actually  learning  (and  thereby  attempting  to  transfer)  through  the 
material.  If,  for  e.xample,  they  are  learning  to  solve  problems  by  focusing  on  discrete 
surface-features  of  the  problem  (as  in  often  the  case  in  JSEP)  "transferability"  of  problem 
solving  skills  would  naturally  be  unlikely.  (For  instance,  inJSEP  students  are  taught  to 
solve  word  problems  by  matching  up  letters  of  quantitv'  words  with  formulas.  If  they  are 
then  asked  to  solve  problems  in  which  the  letters  or  quantities  are  not  immediately 
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evident,  this  learning  becomes  impossible  to  transfer.)  On  the  other  hand,  if  materials 
enable  students  to  learn  something  besides  a  surface-feature  dependent  approach  to  a 
problem  --  for  example,  what  the  "quantities”  in  a  problem  are,  or  how  to  apply  a  formula 
to  problems  that  may  appear  different  on  the  surface  --  then  the  possibility  of  trans¬ 
ferability  increases.* 

Specific  Comments:  On-line  Lesson 

The  lesson  begins  with  an  introduction  stating  the  purpose  of  the  lesson  ("to  learn 
more  effective  ways  of  problem  solving  and  how  to  identify  different  types  of  word 
problems").  The  focus  in  the  second  part  of  this  stated  purpose  is  on  math  word  proble¬ 
ms.  The  lesson  introduces  the  "4  C’s  of  Problem  Solving:  Clarify,  Come  up  with.  Carry 
out,  and  Check,"  and  defines  each  one.  The  student  is  told  that  the  4  C’s  will  help  with 
understanding  math  word  problems,  but  is  not  told  how.  Then,  the  lesson  states  that  the 
4  C’s  can  be  used  to  solve  any  type  of  problem,  not  just  math  word  problems.  It  appears 
that  the  4  C’s  in  this  module  are  the  same  as  the  4  C’s  in  the  Reading  Strategies  lesson 
reviewed  by  Kern.  This  raises  the  interesting  question,  are  math  problem  solving  and 
reading  really  so  similar  that  they  can  be  accomplished  using  the  same  strategy?  A 
second,  more  likely,  implication  is  that  the  4  C’s  "strategy"  is  so  general  as  to  be  of 
limited  use  in  meeting  the  requirements  of  specific  tasks  or  skills. 

In  fact,  the  difficulty  with  the  4  C’s  approach  is  that  it  is  not  sufficiently  "strateg¬ 
ic.”  It  is  not  really  a  problem  solving  strategy,  but  a  general  guideline  for  approaching 
problems.  A  strategy  produces  a  solution  to  a  specific  kind  (or  kinds)  of  problem,  but 
using  the  4  C’s  alone  does  not  produce  a  solution.  The  illustration  problem  -  how  to 
move  prisoners  -  has  nothing  to  do  with  math  word  problems.  The  4  C’s  would  be  better 
illustrated  in  a  context  relevant  to  that  of  the  lesson  . . .  math  word  problems. 

The  next  part  of  the  lesson  deals  with  identifying  different  types  of  word  problems, 
and  has  nothing  to  do  with  the  4  C’s.  This  is  an  apparent  discontinuity.  This  section  of 
the  lesson  is  accessed  through  a  menu  that  lists  different  types  of  word  problems  -- 
conversion  problems,  percent  problems,  work  problems,  etc.  Each  section  (with  the 


^Perhaps  the  reason  why  much  research  has  shown  "non-transferability”  is  simply 
because  learning  materials  are  structured  around  context  rather  than  around  content. 
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exception  of  the  first  one  on  conversion  problems)  deals  with  a  different  type  of  problem. 
Sections  can  be  taken  in  any  order.  Section  B  on  conversion  problems  begins, 


This  exercise  will  help  you  identify  different  types  of  math  word  problems. 

[The  word  "exercise"  is  inappropriate;  this  is  not  an  exercise,  but  a  section  of  a 

lesson.] 

In  case  the  student  doesn’t  understand  "different  types  of  problems,"  the  lesson 
illustrates  the  notion  of  "type"  with  different  types  of  restaurants.  [The  notion  of  "type" 
is  hardly  likely  to  be  a  problem  for  students.]  The  lesson  then  proceeds  to  list  all  the 
different  types  of  word  problems.  This  appears  to  be  misplaced.  The  various  types  of 
problems  should  be  listed  and  described  in  the  introductory  part  of  the  lesson,  not  in  this 
section  on  conversion  problems.  The  section  then  moves  into  a  discussion  of  conversion 
problems: 

A  conversion  problem  is  one  which  asks  you  to  change  one  way  of  measuring 

something  to  another  way  of  measuring.  For  example, 

How  many  inches  are  in  1  1/2  yards? 

Don’t  try  to  figure  out  the  answer.  Just  notice  that  this  is  a  conversion 

problem  because  it  is  asking  you  to  change  one  measurement  scale  to  another. 

["Scale”  is  inappropriate  in  this  context;  the  student  is  not  changing  "scales,"  but  changing 
units.] 

Examples  of  conversion  problems  are  then  given. 

Subsequent  sections  on  percent  problems,  work  problems,  etc.,  follow  the  same 
format.  Practice  items  present  different  types  of  problems  and  require  the  student  to 
identify  the  problem  type. 

The  difficulty  here  is  that  identifying  the  "type"  of  problem  is  only  the  first  step 
toward  solving  it.  Other  instruction  is  needed;  yet,  when  students  reach  the  end  of  the 
on-line  lesson  they  are  told  that  they  have  completed  the  material  on  problem-solving,  and 
are  not  referred  to  the  subsequent  paper-based  lessons. 
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The  Paper-based  Lessons 


Due  to  their  length  (157  pages  total)  and  because  we  were  told  that  students  do  not 
use  them,  these  lessons  were  not  reviewed  systematically.  However,  a  few  observations 
emerged  from  a  cursory  review; 

•  Lesson  3  on  "Goals  and  Diagrams"  begins  by  telling  the  soldier  that  he  or  she 
will  learn  how  to  label  and  diagram  a  math  word  problem.  The  soldier  is  told 
that  labelling  and  diagramming  will  "help  you  work  through  each  of  the  problem 
solving  4  C’s."  (page  3.)  Yet  no  definition  or  description  of  labelling  and 
diagramming  is  ever  given.  The  lesson  even  takes  the  soldier  through  the  first 
step  of  "labelling  and  diagramming"  without  ever  describing  what  it  is. 

•  The  four  steps  to  labelling  and  diagramming  are  impossible  to  perform  in  order: 
one  cannot  do  step  1  (get  rid  of  useless  information)  until  step  2  (identify  and 
label  the  problem  goal)  has  been  completed.  In  other  words,  how  can  one  know 
what  information  is  useless  until  one  knows  what  one  is  looking  for? 

•  Lesson  4  begins  with  illustrations  whose  aim  is  simply  to  "show  how  complex 
word  problems  can  be."  Despite  reassurances  that  "you  will  be  able  to  solve 
problemsiike  these."  it  is  probably  not  a  good  idea  to  begin  a  lesson  by 
resurrecting  student  anxieties. 

•  The  purpose  of  lesson  5  is  to  show  the  student  how  to  do  the  fourth  step  in 
problem  solving  -  "checking  your  answer."  This  returns  to  the  4  C’s,  implying 
that  the  first  three  of  them  have  been  taught.  But  the  stated  purposes  of 
lessons  3  and  4  have  nothing  to  do  with  the  4  C’s. 

Summary  Comments 

There  is  an  apparent  confusion  throughout  the  module  between  the  4  C's  as  a 
problem  solving  strategy,  and  the  unnamed  "strategy"  introduced  in  lessons  3  through  5 
that  involves  the  steps  of  "labelling  and  diagramming"  and  "selecting  sub-goals."  These 
two  approaches  are  not  integrated;  in  fact,  they  make  the  module  appear  confused  and 
disorganized. 
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It  is  doubtful  that  knowing  the  ”4  Cs”  or  how  to  identify  types  of  problems  provides 
much  specific  help  to  students  in  actually  solving  word  problems.  My  impression  is  that 
the  material  in  lesson  3,  though  of  slightly  greater  specificity,  risks  bogging  the  student 
down  --  particularly  the  section  on  diagramming  -  rather  than  providing  him  or  her  with 
easy  to  use  tools  for  problem  solving.  No  clear,  succinct  picture  of  how  the  strategies 
discussed  can  be  used  to  solve  a  problem  from  beginning  to  end  is  ever  given;  rather, 
each  lesson  deals  with  just  one  aspect  of  the  problem  solving,  and  it  is  very  difficult  to 
see  how  these  parts  fit  together  as  a  whole  when  solving  an  actual  problem.  No  applica¬ 
tions  to  problems  presented  in  JSEP  lessons  are  made.  In  sum,  it  is  unlikely  that  the  4 
C’s  (and  the  additional  "strategies"  taught  in  the  paper  lessons)  are  specific  enough  and 
conceptually  rich  enough  to  help  students  solve  problems  more  efficiently. 


Instructor’s  Manual 


Purpose 

The  overall  purpose  of  this  manual  is  not  clear.  At  no  point  is  the  purpose  stated; 
long  descriptive  sections  about  the  JSEP  program  are  intermingled  with  sections  that 
appear  to  be  telling  the  reader  how  to  accomplish  a  particular  task,  leading  to  a  rather 
confusing  presentation  of  te.xtual  material.  Is  the  purpose  of  the  manual  to  act  as  a 
training  tool  for  new  instructors?  Or  is  it  meant  to  be  a  "how  to"  manual  for  specific 
troubleshooting  purposes?  Or  is  it  supposed  to  provide  a  descriptive  overview  of  JSEP  as 
a  program?  If  all  three  functions  are  intended,  these  ought  to  be  clearly  marked  and 
separated  from  one  another. 

An  overall  menu  or  chart  describing  the  basic  parts  of  JSEP  and  how  they  are  set  up 
is  needed  at  the  outset,  in  order  to  provide  some  contextual  understanding  for  what  JSEP 
is  all  about.  Instead,  the  manual  goes  into  detail  about  schedules,  instructor  respon¬ 
sibilities,  benefits,  etc.,  without  at  all  describing  the  basic  components  of  the  JSEP 
program.  Especially  if  the  manual  is  intended  to  act  as  a  training  tool,  it  needs  much 
clearer  organization  and  structuring,  with  detailed  step-by-step  instructions  at  each  stage 
of  involvement  with  the  JSEP  program. 
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The  way  the  manual  is  set  up  now  makes  it  impossible  to  find  information  without 
going  through  a  lot  of  page-turning  and  random  searching.  Instead  of  presenting  a  clear 
description  of  instructors’  roles  in  the  JSEP  classroom,  for  example,  the  manual  presents  a 
"skill  test"  and  "daily  schedule."  These,  however,  are  inadequate  substitutes;  they  do  not 
give  a  clear  picture  of  what  roles  instructors  actually  play  in  JSEP  classes.  Moreover,  it 
does  not  make  much  sense  to  describe  these  until  the  bigger  points  about  how  JSEP  is  set 
up  have  been  made.  Locating  information  about  instructors’  roles  as  well  as  components 
and  functions  of  JSEP  could  be  much  simpler  if  the  manual  were  reorganized  around  clear 
objectives  and  descriptions  that  support  them. 

Also,  comments  that  are  intended  to  be  descriptive  need  to  be  separated  from  those 
that  are  "prescriptive,"  or  that  indicate  a  course  of  action  or  a  procedure  to  be  followed. 
For  example,  the  reader  is  told  that  there  is  "very  minimal  instructor  involvement"  during 
most  of  the  JSEP  classroom  time.  This  certainly  does  not  reflect  the  actual  experience  of 
most  JSEP  instructors;  nor  is  it  a  very  good  recommended  course  of  action.  This 
confusion  of  levels  of  writing  detracts  from  the  authority  of  the  manual. 

Tone  and  Style 

Another  major  area  of  concern  about  the  manual,  particularly  in  the  beginning 
section,  is  tone  and  style.  The  initial  section  presents  the  typical  instructor  as  an 
individual  who  is  unfamiliar  with  computers,  dislikes  them,  resists  them,  and  feels  a  lack 
of  confidence  around  them.  This  is  an  unfounded  stereotype  that  many  instructors  may 
resent.  The  tone  of  this  section  does  nothing  to  alleviate  such  attitudes  even  if  they  do 
exist;  it  reinforces  them  or  may  actually  help  to  foster  them.  Condescending  references 
to  "instructor  fears"  and  insecurities  make  a  negative  impact  on  the  reader  and  establish  a 
tone  that  is  inappropriate  for  a  manual  whose  purpose  presumably  is  to  encourage,  guide, 
and  instruct. 

As  an  introduction  to  JSEP,  this  section  should  emphasize  that  the  instructor  is  not 
merely  an  adjunct  to  the  success  of  JSEP  instruction,  but  a  crucial  component.  This 
would  address  instructors’  concerns  in  a  much  more  direct  manner.  Our  interviews  and 
surveys  found  that  JSEP  instructors  often  feel  that  they  have  less  contact  with  their 
students  than  before,  and  that  they  have  a  harder  time  keeping  track  of  students’ 
progress.  The  belief  that  their  role  may  become  entirely  a  peripheral  one  ought  to  be 


countered  here  with  a  strong  statement  to  the  contrary,  emphasizing  how  JSEP  can  allow 
instructors  to  work  more  closely  with  students  in  a  more  collegial  fashion.  Such  an 
approach  would  be  far  preferable  to  the  stereotypical  "computer-phobic"  image  of  instruc¬ 
tors  actually  presented  in  the  manual. 

Organization 

This  is  a  major  problem  in  the  manual  that  occurs  on  many  different  levels,  from 
overall  organization  of  sections  and  subsections  to  specific  phrasing  within  paragraphs  and 
premature  references  to  JSEP  terms  that  have  not  been  defined. 

This  problem  becomes  particularly  evident  in  section  II.  Sections  on  "benefits  to 
commanders"  and  "history"  seem  inappropriate  here;  discussion  of  "benefits"  should  either 
come  in  Chapter  I  or  in  a  final  concluding  section.  "History"  likewise  ought  to  occur  in  a 
section  describing  the  overall  orientation  of  the  JSEP  program,  perhaps  in  a  separate 
"descriptive"  portion  of  the  manual  in  a  preliminary  orientation  chapter.  Within  the 
"history"  section,  a  flow  chart  on  the  soldier  management  system  (SMS)  suddenly  appears, 
and  seems  totally  out  of  place.  The  SMS  has  not  even  been  described  yet;  moreover,  it 
makes  no  sense  to  introduce  this  in  a  section  on  JSEP  history. 

In  section  III  on  "policies,  procedures,  and  report  forms"  the  reader  is  led  through 
all  the  "supporting"  aspects  of  JSEP  without  first  having  been  given  any  idea  how  JSEP 
actually  operates  or  how  to  operate  it.  This  material  would  fit  much  better  after  a 
section  on  how  to  operate  JSEP  on  the  MicroTICCIT  and  PLATO  delivery  systems.  Also, 
the  first  chapter’s  material  on  scheduling  and  personnel  policies  ought  to  be  combined 
with  this  policy  material;  separating  the  two  only  adds  to  confusion.  Again,  the  tone  in 
this  section  is  rather  "teachery"  at  times  and  inappropriate  for  a  professional-level 
publication. 

Section  IV  on  the  PLATO  system  is  very  confusing.  Such  procedures  as  loading  the 
JSEP  program,  creating  a  sign-on.  creating  the  soldier  demographic  record,  and  using 
instructor  options  are  presented  in  a  way  that  makes  it  very  unclear  for  the  reader  when 
description  of  the  function  ends  and  actual  "how  to"  information  begins.  The  section 
lacks  coherence  and  logical  organization.  It  was  impossible  to  tell  the  difference,  for 
example,  between  what  is  shown  on  the  terminal  screen  and  what  the  user  must  do.  In 
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the  "Creating  a  sign-on"  section,  for  example,  the  term  sign-on  is  never  defined.  The 
section  becomes  confusing  because  of  two  apparent  procedures  of  signing  on  --  one  as 
instructor,  and  one  as  student.  Moreover,  in  this  section,  the  reader  is  suddenly  given  a 
description  of  the  process  for  printing  a  file.  This  seems  to  have  little  to  do  with  the 
sign-on  process  and  simply  does  not  make  sense  here.  For  other  procedures,  such  as 
getting  to  the  "PLATO  facilities  screen"  and  creating  a  soldier  demographic  record,  the 
manual  does  not  give  enough  "how  to"  information,  nor  does  it  give  the  reader  a  clear 
idea  of  the  overall  organization  of  steps  or  procedures  that  must  be  followed  in  using 
JSEP. 


The  section  on  TICCIT  is  much  more  detailed  than  that  given  for  PLATO.  There  is 
a  clearly  labelled  section  on  "logging  on"  for  TICCIT,  as  well  as  sections  on  how  to  "view 
and  edit  soldier  records"  and  "escape  from  a  lesson"  for  TICCIT,  but  not  for  PLATO. 
There  is  more  troubleshooting  information  for  TICCIT  than  for  PLATO.  This  imbalance 
contributes  to  the  overall  impression  of  poor  organization  in  the  manual. 

General  Comments:  Usefulness 

The  manual  does  not  present  information  in  a  readily  accessible  and  understandable 
manner.  Indeed,  without  rather  extensive  prior  e.xperience  with  PLATO  most  instructors 
would  have  a  difficult  time  with  JSEP  if  they  had  to  rely  upon  the  manual  as  their 
primary  information  source.  If  the  manual  is  to  be  used  by  instructors,  it  needs  to  be 
written  from  an  instructor’s  point  of  view  -  perhaps  following  a  question-answer  format, 
or  at  least  offering  some  kind  of  step-by-step  directions  for  crucial  procedures. 

The  language  also  needs  to  be  made  clearer.  Grammatical  errors,  ambiguities,  and 
condescending  or  teachery-sounding  statements  detract  from  the  overall  professionalism  of 
the  manual.  Format  needs  revision;  typographic  conventions  telling  the  reader  when  a 
piece  of  text  is  on-'creen  computer  script,  when  it  represents  something  the  reader  must 
do,  or  when  it  represents  a  key  to  be  pressed  would  be  helpful.  The  manual  would  also 
benefit  from  some  clear  graphic  presentations  that  perhaps  illustrate  what  users  might  see 
on  the  screen  itself,  as  well  as  graphics  illustrating  the  overall  organization  and  specific 
procedures  within  the  JSEP  system. 
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Most  importantly,  perhaps,  the  manual  should  provide  more  positive  encouragement 
for  instructors  in  the  JSEP  program.  Suggestions  for  instructor  involvement,  contribu¬ 
tions,  potential  for  individual  uses  of  JSEP,  and  ideas  on  how  to  enhance  student  involve¬ 
ment  with  JSEP  are  sorely  needed  but  lacking.  What  might  instructors  do  to  facilitate 
student  learning,  other  than  "provide  supplementary  materials?"  As  written,  the  manual  is 
dry  and  lackluster,  something  to  be  kept  on  the  shelf  rather  than  something  to  be  kept 
on  the  desk  and  made  an  integral  part  of  an  instructor’s  classroom  activity. 


The  SMS  —  Features  and  Functions  Guide 

Problems  similar  to  those  found  in  the  Instructor’s  Manual  are  also  present  in  the 
SMS  guide.  The  purpose  of  this  document  needs  to  be  made  explicit,  as  well  as  the 
relationship  between  the  SMS  guide  and  the  Instructor’s  Manual.  How  are  these  to  be 
used?  It  would  be  far  more  convenient  if  they  could  be  combined,  thus  eliminating  what 
is  at  present  considerable  redundancy.  Is  this  guide  supposed  to  be  a  source  of  "how-to" 
information?  If  so,  it  needs  restructuring  and  a  much  clearer  organization,  along  with  a 
subject  index  so  that  users  can  find  the  information  they  need. 

The  guide  presents  no  clear  picture  at  any  point  of  just  what  the  SMS  is.  The  flow 
chart  describing  how  soldiers  move  through  JSEP  (which  appears  seven  times  in  all)  just 
doesn’t  do  the  job;  it  is  not  the  SMS.  A  clear  presentation  of  the  various  aspects  of 
components  of  the  SMS  is  needed  at  the  outset.  Options,  functions,  components,  and 
procedures  appear  jumbled  together;  as  it  stands  it  is  impossible  to  disentangle  "how  to's." 
"must  do’s,"  and  "can  do’s"  from  one  another.  JSEP  terms  (such  as  "soldier  record," 

"soldier  primary  menu,"  "soldier  performance  data,”  "soldier  profile,”  "statistics  option")  are 
used  without  definition,  adding  to  the  confusion.  In  some  cases  it  appears  that  two 
different  terms  in  fact  refer  to  the  same  thing  -  e.g.,  "soldier  performance  data"  and 
"soldier  record."  This  terminology  needs  to  be  straightened  out,  and  clear  definitions  for 
each  JSEP  term  should  be  given  before  such  terms  are  used  in  the  text. 

Section  headings  lack  parallel  structure,  and  seem  to  jump  randomly  from  one  topic 
to  another.  The  first,  "Instructor  Options,"  does  not  describe  instructor  options;  it  seems 
merely  to  draw  attention  to  the  fact  that  there  are  things  called  instructor  options  in 
JSEP.  What  these  are,  however,  is  never  stated.  The  ne.xt  heading,  "Registration."  leads 
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the  reader  to  expect  that  the  registration  process  will  be  described;  in  fact,  the  section 
never  provides  this  information.  It  simply  tells  the  reader  that  a  soldier  needs  to  be 
registered  and  that  demographic  information  needs  to  be  entered  at  some  point,  but  never 
how  to  go  about  doing  this.  It  also  contains  information  on  the  "bases  for  a  soldier’s 
prescription"  which  does  not  make  sense  here.  Linkages  between  components  such  as 
prescription,  demographic  information,  etc.,  need  to  be  made  explicit,  as  well  as  the 
processes  for  accomplishing  such  tasks  as  registration. 

The  next  section  heading,  "Create  Option,"  is  ambiguous.  (Is  "create"  a  verb  or  a 
noun  modifier  here?)  Does  this  section  describe  a  procedure  that  should  be  performed  or 
a  feature  of  the  SMS?  Without  answering  these  questions,  the  guide  then  returns  to  a 
section  called  "Soldier  Demographic  Information,"  which  is  just  what  the  preceding  section 
seemed  to  be  all  about;  it  thus  appears  out  of  place.  The  text  then  mentions  a  "statistics 
option"  that  is  never  explained. 

Another  major  subheading  appears,  "Primary  Menu  Options,"  accompanied  by  the 
statement  that  this  is  another  "statistics  subsection."  What  is  a  statistics  subsection? 

What  is  a  "soldier  menu  line?"  What  are  "primary  menu  options"  in  the  first  place? 

The  section  on  "prescription  and  sequencing"  is  quite  convoluted.  Drawing  a 
distinction  between  prescribing  and  sequencing  assumes  the  reader  has  a  question  about 
this  in  mind;  in  fact,  since  sequencing  has  never  been  mentioned,  the  reader  does  not 
even  know  what  it  is,  let  alone  need  to  distinguish  it  from  prescribing.  The  section  could 
be  written  much  more  clearly  and  succinctly.  Two  paragraphs  are  used  for  what  could  be 
said  in  one  or  two  sentences.  Once  again,  the  flow  chart  does  not  explain  anything. 

Overall,  categories  for  the  types  of  data  collected  by  the  SMS  are  confusingly 
presented,  perhaps  in  part  because  of  inconsistent  terminology  and  terms  that  have  not 
been  adequately  explained.  The  various  components,  features,  functions,  options,  and 
procedures  used  in  JSEP  need  disentangling,  and  clearer  organization  is  essential.  The 
guide  seems  to  have  been  written  by  programmers  for  programmers,  or  for  users  who 
already  have  a  rather  complete  idea  of  what  the  SMS  entails  and  how  to  go  about  using 
it.  Whatever  the  purpose  of  these  materials  may  be.  they  .simply  will  not  get  much  use  in 
the  field  if  they  are  not  substantially  revised  and  rewritten  from  a  user’s  point  of  view. 
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The  JSEP  Test 


The  JSEP  Test  was  developed  by  Florida  State  University  to  serve  as  the  primary 
diagnostic  tool  for  JSEP.  It  was  to  be  administered  to  soldiers  as  they  enrolled  in  the 
JSEP  program  in  order  to  identify  which  prerequisite  competencies  were  already  mastered, 
so  the  soldier  would  not  have  to  waste  time  going  through  material  he  or  she  already 
knew.  Although  it  is  currently  not  being  used,  the  test  was  reviewed  (in  paper  format)  as 
part  of  the  overall  evaluation  of  JSEP  supporting  materials.  Detailed  comments  were 
submitted  to  the  Army  Research  Institute,  recommending  substantial  revisions  if  the  test 
were  to  be  used  as  the  primary  means  of  diagnosing  student  weaknesses.  By  way  of 
illustration  of  some  of  the  major  difficulties  with  the  test,  a  summary  critique  is  presen¬ 
ted  here. 

The  300-item  test  was  divided  into  two  sections,  verbal  and  math.  However,  some 
items  supposedly  testing  verbal  ability  depended  more  on  math  skills  (and  vice-versa)  so 
that  a  clear  distinction  between  verbal  and  math  items  was  not  always  maintained. 

The  major  problems  with  test  items  fell  into  the  following  categories:  (1)  problems 
with  wording  -  awkward  phrasing,  ambiguity,  verbosity,  poor  word  choice;  (2)  problems 
with  item  focus  (i.e.,  the  item  tests  a  skill  or  knowledge  other  than  the  one  intended  - 
for  example,  a  math  item  tests  knowledge  of  math  vocabulary  rather  than  sequencing  skill, 
or  a  reading  item  that  tests  specific  factual  knowledge  rather  than  reading  ability;  (3) 
problems  with  answer  choices  -  e.g.,  more  than  one  possible  answer;  (4)  lack  of  sufficient 
information  to  determine  the  item  answer;  and  (5)  unclear  drawings  or  illustrations.  Of  a 
total  of  180  items  on  the  math  section  of  the  test,  43  were  found  to  be  defective  for  one 
of  the  five  reasons  just  indicated.  Eighteen  items  had  language  problems;  seven  lacked 
essential  information  upon  which  to  base  an  answer;  six  were  misfocused.  Of  the  120 
items  on  the  verbal  portion  of  the  test,  29  were  defective,  with  nine  poorly  worded,  five 
misfocused,  and  six  lacking  information  required  to  answer  the  question. 

For  these  reasons,  it  was  recommended  that  the  test  be  substantially  revised  (and 
shortened)  before  being  used  as  the  JSEP  diagnostic  instrument. 
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Chapter  5.  Summary  of  Findings  and  Recommendations 


This  chapter  summarizes  the  findings  and  recommendations  made  in  previous  chapters 
of  this  volume.  The  criticisms  made  throughout  this  review  may  be  seen  as  specific 
recommendations  for  changes  to  the  JSEP  program.  In  this  chapter,  however,  only  more 
general  findings  and  recommendations  are  discussed. 


Specific  Problem  Areas  in  Lessons 

This  review  identified  IS  different  problem  areas  in  JSEP  lessons  which  fell  roughly 
into  three  categories:  problems  in  conceptual  development,  problems  in  lesson  presenta¬ 
tion  or  delivery,  and  problems  concerning  the  relationship  between  the  short  lesson  and 
the  long  lesson.  Summary  comments  and  recommendations  for  each  problem  area  follow. 

Category  I.  Conceptual  Problems 

Problem  area  1:  The  lesson  primarily  teaches  skills  that  are  only  peripherally  or 
superficially  related  to  the  targeted  skill.  This  was  one  of  the  major  weaknesses  of  the 
instnictional  approach  taken  in  JSEP  lessons.  Fifty-three  percent  of  the  lessons  reviewed 
showed  evidence  of  this  problem.  In  such  lessons,  students  were  taught  skills  that 
focused  on  manipulation  or  identification  of  surface  features  of  a  problem  or  situation, 
instead  of  being  taught  the  underlying  targeted  skill  of  the  lesson.  It  is  recommended 
that  such  lessons  be  re-written  to  eliminate  exclusive  focus  on  peripheral  skills,  so  that 
they  in  fact  teach  the  skill  they  intend  to. 

Problem  area  2:  The  lesson  uses  an  inappropriate  rule  or  step-based  approach  to 
instruction.  This  weakness  was  found  in  45  percent  of  the  lessons  reviewed.  Students 
were  taught  to  solve  problems  by  following  lists  of  steps  or  rules,  instead  of  by  acquiring 
an  adequate  conceptual  understanding  of  the  process.  Because  of  the  lack  of  focus  on 
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underlying  concepts,  steps  or  rules  often  had  to  be  presented  for  each  specific  problem 
case,  thus  increasing  the  amount  of  memorization  or  rote  work  the  student  was  required 
to  do.  Following  the  steps  could  lead  students  to  successful  performance  of  the  targeted 
skill,  though  without  an  understanding  of  what  they  were  doing  as  they  solved  a  problem. 

In  some  cases,  lists  of  steps  or  rules  led  to  highly  inefficient  use  of  student  energy  and 
time;  targeted  skills  could  be  performed  much  more  easily  if  students  were  given  fewer 
steps  to  follow.  It  is  recommended  that  lists  of  steps  or  rules  be  used  only  when  they 
represent  the  most  efficient  and  easiest  way  to  solve  problems,  and  then  only  accompanied 
by  adequate  concept-based  explanations. 

Problem  area  3:  Explanations  for  processes  or  procedures  are  incomplete  or  lacking. 
This  difficulty  was  found  in  20  percent  of  the  lessons  reviewed.  Although  students  were 
told  to  follow  a  procedure  in  order  to  solve  a  problem,  reasons  behind  the  process  or 
explanations  for  it  were  lacking.  It  is  recommended  that  such  explanations  be  provided, 
for  they  ultimately  lead  to  more  effective  learning  of  lesson  material,  and  possibly  to 
learning  that  is  more  transferable  to  other  non-lesson  related  situations. 

Problem  area  4:  Explanations  related  to  the  main  targeted  skill  in  the  lesson  are 
misleading,  incomplete  or  lacking.  This  was  the  most  common  problem  in  the  lessons 
reviewed,  appearing  in  60  percent  of  the  cases.  Misleading  definitions  of  basic  terms  led 
to  confusion  in  some  lessons;  in  others,  the  targeted  skill  was  inadequately  defined, 
leading  to  weaknesses  in  subsequent  instruction.  These  lessons  could  be  improved  by 
providing  accurate  and  complete  definitions  and  explanations  for  all  basic  terms  and  skill 
objectives. 

Problem  area  5:  The  lesson  lacks  an  adequate  introduction  or  context  for  skill 
instruction.  Twenty-three  percent  of  the  lessons  reviewed  showed  this  weakness. 

Providing  an  introduction  or  context  for  instruction  shows  students  why  a  skill  is  impor¬ 
tant,  and  gives  students  an  idea  where  the  lesson  is  heading.  It  allows  them  to  see  how 
parts  (or  sub-skills)  are  related  to  the  whole  (or  major  skill  objective.)  In  most  lessons, 
this  deficiency  could  be  remedied  by  providing  clear  overviews  of  skills  to  be  taught 
before  instruction  is  begun.  This  information  ought  to  be  accompanied  by  some  statement 
about  its  usefulness  to  the  soldier. 
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Problem  area  6:  The  lesson  has  organizational  problems.  This  difficulty  was  found 
in  18  percent  of  the  lessons  reviewed.  It  was  easiest  to  analyze  in  paper-based  lessons, 
since  ail  materials  were  readily  available  to  the  reviewer  at  one  time.  Most  often, 
organizational  difficulties  appeared  when  practice  items  were  placed  in  sections  to  which 
they  did  not  refer,  or  lesson  headings  and  subsections  were  inconsistent,  redundant,  or 
simply  confusing.  In  such  lessons,  re-organization  and  careful  checking  of  section 
coherence  and  consistency  is  suggested. 

Problem  area  7:  Lesson  instruction  relies  upon  inappropriate  comparisons  or  confuses 
one  skill  with  another.  In  25  percent  of  the  lessons  reviewed  difficulties  were  found 
related  to  the  use  of  inappropriate  comparisons  or  skill  confusion.  This  difficulty  is 
illustrated,  for  example,  in  using  a  temperature  scale  to  illustrate  number  quantities,  or 
equating  the  ability  to  notice  decimal  point  position  with  being  able  to  tell  the  place 
value  of  a  number.  Instruction  in  such  cases  could  be  improved  by  use  of  more  accurate 
comparisons,  and  focus  on  skills  that  are  in  fact  readily  interchangeable. 

Problem  area  8:  The  lesson  contains  logical  inconsistencies  or  contradictions.  This 
difficulty  was  found  in  only  ten  percent  of  the  lessons  reviewed.  These  lessons  had 
circular  presentations  of  material  or  explanations  that  led  to  contradictions  within  sections 
or  across  parts  of  lessons.  Careful  re-writing  with  attention  to  logical  consistency  of 
arguments  and  materials  is  recommended. 


Category  II.  Presentation  and  Delivery  Problems 

Problem  area  9:  Lesson  practice  and/or  test  items  do  not  provide  adequate  informa¬ 
tion  about  the  answer  format  required,  or  answers  to  questions  depend  too  much  on 
mechanical  performance  factors  such  as  manipulating  computer  keys.  In  20  percent  of 
lessons  reviewed,  mechanical  format  problems  interfered  with  answering  practice  or  test 
questions.  In  some  lessons  answers  had  to  be  in  a  format  that  was  not  indicated.  (This 
problem  arose  most  often  on  the  lesson  prettest.  Once  students  took  the  short  lesson, 
however,  they  became  aware  of  the  required  formats  since  practice  items  use  the  same 
format  as  test  items.  However,  it  is  still  important  that  test  items  state  specifically  the 
answer  formats  required,  and  not  just  rely  on  students’  short  lesson  experience.)  In  other 
lessons,  answers  depended  upon  manipulating  the  on-screen  cursor,  or  required  the  student 
to  do  too  much  detailed  typing  that  could  easily  lead  to  errors.  Such  items  need  to  have 
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answer  formats  specified,  or  else  they  need  to  be  reprogrammed  to  accept  a  larger  variety 
of  formats.  Those  items  that  depend  inordinately  upon  typing  skill  should  be  revised. 


Problem  area  10:  The  les.son  has  problems  with  language:  ambiguities.  redundancie«; 
inconsistencies,  or  errors.  Forty-eight  percent  of  the  lessons  reviewed  manifested  some 
type  of  language  problem,  ranging  from  minor  typographical  errors  to  confusing  definitions 
of  basic  terms.  These  problems  were  among  those  most  obvious  to  the  JSEP  user,  and 
among  those  that  appeared  to  detract  from  users’  sense  of  satisfaction  with  the  program. 

Problem  area  11:  Practices  or  examples  are  inappropriate  for  the  given  lesson  skill 
objective.  This  problem  was  detected  in  30  percent  of  the  lessons  reviewed.  In  some 
cases,  practices  did  not  match  lesson  material;  in  others,  they  gave  students  practice  in 
skills  only  minimally  related  to  the  objective  of  the  lesson.  For  example,  a  lesson  on 
measuring  angles  might  contain  a  long  practice  item  sequence  focused  solely  on  naming 
parts  of  angles.  Practice  items  should  be  re-written  to  maintain  adequate  focus  on  the 
target  skill  of  the  lesson. 

Problem  area  12:  Feedback  to  practice  items  is  inappropriate.  Twenty  percent  of 
the  lessons  reviewed  had  practice  items  whose  feedback  was  either  misleading  or  inap¬ 
propriate.  In  some  cases  the  feedback  assumed  the  student  had  made  an  error  he  or  she 
may  not  have  made.  In  other  cases,  the  feedback  did  not  appear  to  match  the  question. 

It  is  recommended  that  feedback  to  such  questions  be  re-written  so  that  it  does  not 
mislead  the  student  into  thinking  he  or  she  made  a  certain  type  of  error  that  in  fact  was 
not  made. 

Problem  area  13:  The  lesson  contains  confusing  diagrams  or  illustrations.  This 
difficulty  was  found  in  only  13  percent  of  the  lessons.  In  some  cases,  it  was  difficult  to 
read  parts  of  illustrations  because  labels  were  misplaced  or  ambiguously  positioned,  or 
because  the  illustration  was  extremely  detailed.  Recommended  changes  include  positionins 
all  labels  so  that  they  can  be  clearly  matched  with  indicated  points  on  the  illustration. 

Problem  area  14:  The  lesson  uses  inappropriate  instmctional  methods.  This  problem 
was  present  in  18  percent  of  the  lessons  reviewed.  For  example,  before  giving  any 
instruction  in  the  target  skill,  some  lessons  posed  a  test  question  the  student  was  sure  to 
miss  "  apparently  to  demonstrate  to  the  student  just  how  much  he  or  she  did  not  know. 


In  other  cases,  the  lesson  would  ask  the  student  if  he  or  she  knew  certain  material,  and 
then  proceed  to  explain  that  same  material  even  if  the  student  answered  "yes."  Such 
inappropriate  instructional  strategies  should  be  eliminated  from  lessons. 

Category  DI.  Problems  in  Short  and  Long  Lesson  Compatibility  and  Tests 

Problem  area  15:  For  a  given  prerequisite  competency,  short  and  long  lessons  use  a 
similar  or  identical  approach  to  instruction.  This  problem  was  present  in  28  percent  of 
the  lessons  reviewed.  Despite  the  intention  of  JSEP  lesson  design  to  focus  on  review  in 
the  short  lesson  and  instruction  in  the  long  lesson,  in  reality  these  lessons  did  not 
significantly  differ  from  each  other  in  their  instructional  approach.  Though  long  lessons 
did  tend  to  provide  more  "walk-through"  of  basic  skills  underlying  a  given  targeted 
competency,  when  it  came  to  the  targeted  skill  itself,  sometimes  whole  sections  of 
material  from  the  short  lesson  were  repeated  verbatim  in  the  long  lesson.  Such  long 
lessons  did  not,  in  fact,  provide  any  instruction  significantly  different  from  the  remedia¬ 
tion  of  short  lessons.  It  is  recommended  that  in  order  to  make  long  lessons  more 
effective,  instructional  approaches  different  from  those  taken  in  the  short  lessons  be  used. 

Problem  area  16:  Short  and  long  lessons  do  not  match  in  content.  In  20  percent  of 
the  lessons  reviewed,  short  and  long  lessons  taught  different  things.  In  at  least  one  case, 
information  presented  in  the  long  lesson  contradicted  information  given  in  the  short 
lesson;  in  other  cases,  students  were  simply  taught  related,  though  different,  skills.  It  is 
recommended  that  short  and  long  lessons  be  reviewed  for  instances  of  content  mismatch. 

Problem  area  17:  Test  items  do  not  match  lesson  material.  This  difficulty  was 
present  in  23  percent  of  the  lessons  reviewed.  In  some  cases,  information  not  taught  in 
the  lesson  is  necessary  in  order  to  answer  test  questions.  In  other  cases,  test  items 
focused  on  skills  that  were  not  those  of  the  lesson  objective.  .All  instances  of  test  item 
mismatch  should  be  corrected. 

Problem  area  18:  Test  items  are  confusing  or  ambiguous.  Twenty-three  percent  of 
the  lessons  reviewed  contained  at  least  one  such  problem  test  item.  In  some  cases,  the 
confusion  was  due  to  language;  in  others,  to  misleading  drawings  or  illustrations.  It  is 
recommended  that  all  such  items  be  re-written  or  eliminated. 
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General  Curricular  Concerns 


Aside  from  the  above  noted  specific  lesson  weaknesses  in  the  JSEP  curriculum,  the 
review  identified  more  general  areas  in  which  there  is  some  room  for  improvement. 
Although  interactiveness  of  the  program  was  high,  with  users  constantly  required  to 
answer  questions  and  make  choices  while  taking  lessons,  there  appeared  to  be  a  certain 
inflexibility  in  the  program.  This  derived  from  the  fact  that  though  students  could  decide 
the  order  of  lessons  they  took,  or  in  some  cases  the  order  of  sections  within  a  lesson, 
more  responsible  choices  --  such  as  deciding  to  take  a  lesson  over  again,  repeat  a  section 
in  a  lesson,  skip  a  section  they  felt  they  did  not  need,  or  select  a  lesson  to  fit  a  limited 
time  period  --  were  not  possible.  Yet  students  repeatedly  stressed  that  they  were  willing 
and  able  to  make  these  choices. 

A  second  more  general  concern  relates  to  the  level  of  skills  actually  taught  by  the 
JSEP  curriculum.  Although  each  lesson  was  designed  to  review  or  teach  a  specific 
prerequisite  competency,  the  competency  (or  skill  objective)  was  frequently  broken  down 
into  lower  level  component  skills  that,  when  added  together,  did  not  adequately  represent 
the  original  targeted  competency.  JSEP  lessons  thus  tended  to  teach  lower-order  skills 
such  as  identification,  recognition,  pattern  matching,  element  manipulation  --  rather  than 
higher-order  skills  such  as  association,  abstraction,  synthesis,  and  evaluation.  Yet  these 
higher  order  skills  were  often  essential  to  learning  the  stated  competencies. 

.A.  third  issue  related  to  the  focus  on  lower  order  cognitive  skills  concerns  the 
tendency  of  JSEP  lessons  to  encourage  students  to  remain  grounded  in  their  thinking,  and 
to  approach  problems  or  situations  only  in  the  most  literal  way.  Students  were  not  taught 
lesson  material  in  such  a  way  as  to  encourage  transfer  or  application  to  other  related 
contexts;  instead,  they  were  taught  to  follow  lists  of  steps  or  rules  for  each  separate  case 
or  particular  type  of  problem.  In  general  they  were  not  given  practice  in  applying 
learning  to  novel  contexts,  in  abbreviating  steps,  or  in  approaching  problems  from  other 
than  a  focus  on  surface  features  of  the  problem. 
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JSEP  Supporting  Materials  Review 


The  JSEP  Instructor’s  Manual,  the  Soldier  Management  Svstem:  Features  and 
Functions  Guide,  the  Learning  Strategies  lesson  on  problem  solving,  and  the  JSEP  test 
were  reviewed.  The  two  manuals  were  difficult  to  understand  and  use  because  of  or¬ 
ganization  problems,  language,  and  generally  unclear  sense  of  purpose.  Unless  users  had 
prior  knowledge  of  the  JSEP  program,  the  manuals  would  be  veiy  difficult  to  use. 

The  Problem  Solving  Module,  one  of  five  Learning  Strategies  lessons,  was  found  to 
have  a  discontinuity  in  its  design,  with  the  on-line  portion  of  the  lesson  failing  to 
establish  any  connection  to  the  paper-based  lessons  that  followed  it.  Information  from 
sites  indicated  that  only  the  on-line  portion  of  the  lesson  was  used.  However,  the  on-line 
portion  did  not  teach  students  how  to  solve  problems  more  efficiently,  but  only  how  to 
identify  different  types  of  problems. 

The  JSEP  test  was  found  to  have  numerous  items  that  were  defective  due  to 
problems  with  language,  item  choices,  focus,  or  lack  of  information  necessary  to  answer 
the  item  correctly. 


Recommendations  for  Curriculum  Improvement 

Suggestions  for  curriculum  revision  flow  from  the  above  identified  problem  areas.  In 
addition  to  the  obvious  need  to  eliminate  all  errors,  the  following  recommendations  are 
made: 

1.  Re-write  lessons  to  reduce  a  focus  on  steps  or  rules,  or  supplement  lists  of  steps 
with  clear  explanations  of  underlying  concepts  or  ideas. 

2.  Include  problems  in  each  lesson  that  allow  and  encourage  students  to  practice 
applying  skills  learned  in  related,  novel  contexts. 

3.  Provide  more  opportunity  for  students  to  practice  higher-order  skills.  Reduce 


76 


number  of  lessons  and  number  of  practice  items  in  lessons  that  are  concerned 
solely  with  rote,  identification-level  tasks. 

4.  Re-write  lessons  to  provide  explanatory  contexts  for  all  lesson  material. 

5.  Revise  and  substantially  reduce  the  length  of  paper-based  lessons,  or  make  these 
optional  for  use  during  computer  down-time  only. 

6.  Re-write  long  lessons  so  that  they  do  not  repeat  material  from  short  lessons,  but 
take  a  different  insiiuctional  approach.  Also,  reduce  focus  on  component  lower- 
order  skills  such  as  identification  in  long  lessons,  and  substitute  with  more 
varied  concept-level  explanations. 

7.  Allow  students  more  freedom  to  repeat  lessons  or  sections  of  lessons,  skip  lesson 
sections  that  are  not  useful,  and  in  general,  to  choose  the  lessons  they  take. 

8.  Allow  students  to  see  their  errors  on  tests. 

9.  Revise  manuals  to  make  them  more  helpful  and  user-friendly. 

If  such  changes  are  made,  JSEP  can  undoubtedly  make  the  contributions  to  education 
and  job  preparation  that  it  was  designed  to  make.  It  remains  for  students,  JSEP  instruc¬ 
tors,  program  administrators,  and  designers  to  work  ever  more  closely  together  to  reach 
these  2oals. 
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